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(54) OPTICAL DISC FOR RECORDING HIGH RESOLUTION AND NORMAL IMAGE, OPTICAL DISC 
PLAYER, OPTICAL DISC RECORDER, AND PLAYBACK CONTROL INFORMATION 
GENERATOR 



(57) The present invention has an objective of real- 
izing compatibility of an optical disk having a high reso- 
lution video signal recorded thereon and a system for 
reproducing the optical disk, with a conventional system 
for producing a standard resolution video signal. A high 
resolution video signal is divided by video division 
means into a main signal and a sub signal, and the main 
signal and the sub signal are MPEG-encoded. The 
stream of the main signal and the stream of the sub sig- 
nal are divided into 1 GOP or more of frames. First inter- 
leave blocks 54 each including 1 GOP or more of the 
stream of the main signal and second interleave blocks 
55 each including 1 GOP or more of the stream of the 
sub signal are recorded on an optical disk 1 . A high res- 
olution reproduction apparatus reproduces both the first 
and second interleave blocks to obtain a high resolution 
video output. A non-high quality picture reproduction 
apparatus reproduces only the first or second interleave 
blocks to obtain a standard resolution video output. 
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Description 

TECHNICAL FIELD 

[0001] The present invention relates to an optical 
disk having high quality picture or standard picture 
recorded thereon, and a recording and reproduction 
apparatus for such an optical disk. 

BACKGROUND ART 

[0002] For an optical disk having high quality picture 
recorded thereon and a reproduction apparatus for such 
an optical disk, systems referred to as 480 P and 720 P 
for recording progressive data have conventionally been 
studied. A conventionally known reproduction control 
system for an optical disk uses one MPEG decoder. 
[0003] First, a first problem of the conventional sys- 
tem will be described. When a conventional optical disk 
for high quality picture recording is reproduced by a 
standard reproduction apparatus, normal pictures can- 
not be output. The optical disk for high quality picture 
recording can only be reproduced by a high quality pic- 
ture reproduction apparatus. Accordingly, there is a 
need to produce two types of optical disks having the 
same contents. In other words, the conventional optical 
disk for high quality picture recording is not compatible 
with a standard video reproduction apparatus. Next, 
objectives of the present invention will be described. A 
first objective of the present invention is for providing a 
optical disk for high quality picture recording compatible 
with the standard video reproduction apparatus and a 
reproduction system for such an optical disk. 
[0004] The compatibility herein can be defined as 
the relationship between the conventional monaural 
records and stereo records. That is, a novel 3D optical 
disk or high resolution optical disk according to the 
present invention is output with a standard resolution by 
the existing reproduction apparatus for DVD or the like, 
and output with a high resolution by a novel reproduc- 
tion apparatus according to the present invention. 
[0005] Then, a second problem of the conventional 
system is regarding a reproduction control system. By 
the conventional reproduction control system, one 
stream is reproduced using one decoder. Accordingly, 
in order to connect two streams of a high resolution sig- 
nal seamlessly, i.e., without stopping the movement of 
the video, a complicated system is required. A second 
objective of the present invention is for providing a 
reproduction control for connecting a plurality of 
streams seamlessly by a simple procedure. 

DISCLOSURE OF INVENTION 

[0006] An optical disk reproduction apparatus 
according to the present invention is for reproducing a 
signal recorded on an optical disk. The optical disk has, 
recorded thereon, at least a first video stream repre- 



senting a low frequency component of the video signal 
and a second video stream representing at least a high 
frequency component of the video signal, the first video 
stream includes a plurality of first interleave units and 
5 the second video stream includes a plurality of second 
interleave units, each of the plurality of first interleave 
units includes ml GOPs (where ml is an integer of 1 or 
greater), each of the plurality of second interleave units 
includes m2 GOPs (where m2 is an integer of 1 or 
10 greater). The optical disk reproduction apparatus 
includes a reproduction section for reproducing the first 
video stream and the second video stream recorded on 
the optical disk; a division section for dividing the repro- 
duced first video stream into the plurality of first inter- 
ns leave units and for dividing the reproduced second 
video stream into the plurality of second interleave 
units; a decoding section for decoding the plurality of 
first interleave units to generate a first reproduction sig- 
nal representing the low frequency component of the 
20 video signal and for decoding the plurality of second 
interleave units to generate a second reproduction sig- 
nal representing at least the high frequency component 
of the video signal; a synthesis section for synthesizing 
the first reproduction signal and the second reproduc- 
es tion signal to generate the video signal; and an output 
section for selectively outputting at least one of the first 
reproduction signal, the second reproduction signal, 
and the video signal. The above-described objective is 
achieved by this. 
30 [0007] The plurality of first interleave units may be 
each corresponded to first time information relating to 
reproduction time, and the plurality of second interleave 
units may be each corresponded to second time infor- 
mation relating to reproduction time. 
35 [0008] The optical disk reproduction apparatus may 
further include a reference time signal generation sec- 
tion for generating a reference time signal; a first repro- 
duction control section for controlling the reproduction 
time of the first reproduction signal in accordance with 
40 the difference between the reference time signal and 
the first time information; a second reproduction control 
section for controlling the reproduction time of the sec- 
ond reproduction signal in accordance with the differ- 
ence between the reference time signal and the second 
45 time information; and an adjusting section for adjusting 
the reference time signal so that the reference time sig- 
nal supplied to the first reproduction control section and 
the reference signal supplied to the second reproduc- 
tion control section represent substantially the same 
50 time. 

[0009] The adjusting section may adjust the refer- 
ence time signal based on audio reproduction time 
information representing the time to reproduce an audio 
signal which is to be output in synchronization with the 
55 video signal. 

[0010] The adjusting section may adjust the refer- 
ence time signal based on at least one of first video 
reproduction time information representing the time to 
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reproduce the first reproduction signal and second 
video reproduction time information representing the 
time to reproduce the second reproduction signal. 
[0011] The first reproduction control section may 
control the reproduction time of the first reproduction 
signal by skipping a frame of the first reproduction sig- 
nal or by reproducing a frame of the first reproduction 
signal in repetition. The second reproduction control 
section may control the reproduction time of the second 
reproduction signal by skipping a frame of the second 
reproduction signal or by reproducing a frame of the 
second reproduction signal in repetition. 
[0012] At least one of the first time information and 
the second time information may include at least one of 
a PTS, a DTS and an SCR. 

[001 3] The first reproduction signal may correspond 
to a first pixel number, and the second reproduction sig- 
nal may correspond to a second pixel number, which is 
larger than the first pixel number. The synthesis section 
may include a converter for converting the first repro- 
duction signal into a conversion signal corresponding to 
the second pixel number. The video signal may be 
obtained by synthesizing the conversion signal and the 
second reproduction signal. 

[0014] The optical disk further may have, recorded 
thereon, an identifier representing the first pixel number 
corresponding to the first reproduction signal, and the 
converter may convert the first reproduction signal into 
the conversion signal in accordance with the identifier. 
[0015] The optical disk further may have, recorded 
thereon, an identifier representing the first pixel number 
corresponding to the first reproduction signal. The opti- 
cal disk reproduction apparatus may further include a 
rotation control section for controlling the rotation of the 
optical disk. The rotation control section may control the 
rotation of the optical disk in accordance with the identi- 
fier. 

[0016] The optical disk further may have, recorded 
thereon, an identifier representing that the video signal 
is obtained by encoding a progressive video signal of 24 
frames to 30 frames per second. The output section 
may include a converter for converting at least one of 
the first reproduction signal, the second reproduction 
signal, and the video signal into a frame signal. The out- 
put section may output the progressive video signal of 
60 frames per second by outputting the frame signal in 
an overlapping manner. 

[0017] The optical disk reproduction apparatus may 
further include a buffer memory section for storing the 
plurality of first interleave units and the plurality of sec- 
ond interleave units. The buffer memory section may 
have a capacity which is equal to or greater than an 
amount of data of the GOP or GOPs included in the sec- 
ond interleave units. 

[0018] The buffer memory section may have a 
capacity which is 1 MB or greater. 
[0019] An optical disk according to the present 
invention include, recorded thereon, at least a first video 



stream representing a low frequency component of the 
video signal and a second video stream representing at 
least a high frequency component of the video signal, 
wherein: the first video stream includes a plurality of first 

5 interleave units, the second video stream includes a 
plurality of second interleave units, each of the plurality 
of first interleave units includes ml GOPs (where ml is 
an integer of 1 or greater), and each of the plurality of 
second interleave units includes m2 GOPs (where m2 is 

10 an integer of 1 or greater). The above-described objec- 
tive is achieved by this. 

[0020] The plurality of first interleave units and the 
plurality of second interleave units may be structured so 
that reproduction time of one of the plurality of first inter- 

15 leave units is substantially equal to reproduction time of 
one of the plurality of second interleave units, the one of 
the plurality of second interleave units corresponding to 
the one of the plurality of first interleave units. 
[0021] An optical disk recording apparatus accord- 

20 ing to the present invention includes a dividing section 
for dividing a video signal into a first video signal repre- 
senting a low frequency component of the video signal 
and a second video signal representing at least a high 
frequency component of the video signal; an encoding 

25 section for generating a first video stream by encoding 
the first video signal and for generating a second video 
stream by encoding the second video signals wherein: 
the first video stream includes a plurality of first inter- 
leave units, the second video stream includes a plurality 

30 of second interleave units, each of the plurality of first 
interleave units includes ml GOPs (where ml is an 
integer of 1 or greater), and each of the plurality of sec- 
ond interleave units includes m2 GOPs (where m2 is an 
integer of 1 or greater); a selection output section for 

35 selectively outputting the plurality of first interleave units 
included in the first video stream and the plurality of sec- 
ond interleave units included in the second video 
stream: and a recording section for recording the signal 
output from the selection output section on an optical 

40 disk. The above-described objective is achieved by this. 
[0022] The division section may include a decoder 
for decoding the first video stream and a differential cal- 
culator for calculating a differential between the video 
signal and the signal output from the decoder, and may 

45 output the signal output from the differential calculator 
as the second video signal. 

[0023] The division section may further include a 
first converter for converting the video signal into a first 
conversion signal corresponding to a second pixel 

50 number which is smaller than a first pixel number corre- 
sponding to the video signal, and a second converter for 
converting the signal output from the decoder into a 
second conversion signal corresponding to the first pixel 
number which is larger than the second pixel number 

55 corresponding to the signal output from the decoder. 
The division section may output the first conversion sig- 
nal as the first video signal. The differential calculator 
may calculate the differential between the video signal 
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and the second conversion signal. 
[0024] The recording section may further record on 
the optical disk an identifier representing that the sec- 
ond video signal is output from the differential calcula- 
tor. 5 
[0025] The recording section may further record on 
the optical disk an identifier representing the first pixel 
number corresponding to the video signal. 
[0026] The recording section may further record on 
the optical disk an identifier representing the second 10 
pixel number corresponding to the first video signal. 
[0027] An optical disk recording apparatus accord- 
ing to the present invention includes an input section for 
receiving an encoded first video stream corresponding 
to a first pixel number and an encoded second video 15 
stream corresponding to a second pixel number which 
is different from the first pixel number, wherein the first 
video stream includes a plurality of first interleave units, 
the second video stream includes a plurality of second 
interleave units, each of the plurality of first interleave 20 
units includes ml GOPs (where ml is an integer of 1 or 
greater), and each of the plurality of second interleave 
units includes m2 GOPs (where m2 is an integer of 1 or 
greater); a selection output section for selectively out- 
putting the plurality of first interleave units included in 25 
the first video stream and the plurality of second inter- 
leave units included in the second video stream; and a 
recording section for recording the signal output from 
the selection output section on an optical disk. The 
above-described objective is achieved by this. 30 
[0028] An optical disk reproduction apparatus 
according to the present invention is for reproducing a 
signal recorded on an optical disk. The optical disk has, 
recorded thereon, at least a first video stream including 
a plurality of first GOPs and a second video stream 35 
including a plurality of second GOPs, each of the plural- 
ity of first GOPs includes a plurality of pictures, and 
each of the plurality of second GOPs includes a plurality 
of pictures, The optical disk reproduction apparatus 
includes a reproduction section for reproducing the first 40 
video stream and the second video stream recorded on 
the optical disk; a decoding section for decoding the first 
video stream and the second video stream; and an out- 
put section for selectively outputting the decoded first 
video stream and the decoded second video stream in 45 
accordance with reproduction control information. The 
reproduction control information indicates that after a 
first picture included in a final first GOP among the plu- 
rality of first GOPs included in the first video stream is 
reproduced, a second picture included in a leading sec- so 
ond GOP among the plurality of second GOPs included 
in the second video stream is reproduced, the second 
picture being different from a leading picture of the lead- 
ing second GOP The above-described objective is 
achieved by this. 55 
[0029] The decoding section may start decoding 
the second video stream so that the decoding of the 
second picture has been completed when the reproduc- 
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tion of the first picture is completed. 
[0030] The reproduction control information may 
include information ts1 representing a position of the 
first picture, information ts2 representing a position of 
the second picture, and information tsG representing a 
position of the leading picture of the leading second 
GOP. The decoding section may find a decoding start 
position ta in accordance with expression 
ta=ts1-(ts2-tsG) , and starts decoding the second video 
stream based on the decoding start position ta. 
[0031] The reproduction control information may 
include timing information representing the timing to 
start decoding the leading second GOP so that repro- 
duction completion time of the first picture matches the 
reproduction start time of the second picture. The 
decoding section may start decoding the second video 
stream based on the timing information. 
[0032] The decoding section may omit decoding of 
a picture which is not necessary for decoding pictures 
from the leading picture of the leading second GOP to 
the second picture. 

[0033] The picture which is not necessary may be a 
B picture. 

[0034] The optical disk reproduction apparatus may 
further include a buffer memory section for storing the 
first video stream and the second video stream, and the 
buffer memory section has a capacity which is equal to 
or greater than an amount of data of 1 GOP. 
[0035] The optical disk has the reproduction control 
information recorded thereon. The reproduction section 
may reproduce the reproduction control information 
recorded on the optical disk. 

[0036] The optical disk may further have, recorded 
thereon, an identifier representing whether or not the 
reproduction control information is recorded on the opti- 
cal disk, and when the identifier represents that the 
reproduction control information is recorded on the opti- 
cal disk, the reproduction section may reproduce the 
reproduction control information recorded on the optical 
disk. 

[0037] In a fast reproduction mode, when the sec- 
ond picture is not an I picture, the output section may 
prohibit an I picture included in the leading second GOP 
from being output. 

[0038] The output section may prohibit a part of the 
I picture included in the leading second GOP from being 
output based on I picture reproduction prohibition infor- 
mation. 

[0039] A reproduction control information genera- 
tion apparatus according to the present invention 
includes an input section for receiving a first video 
stream including a plurality of first GOPs and a second 
video stream including a plurality of second GOPs; and 
a generation section for generating reproduction control 
information which represents that after a first picture 
included in a final first GOP among the plurality of first 
GOPs included in the first video stream is reproduced, a 
second picture included in a leading second GOP 
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among the plurality of second GOPs included in the 
second video stream is reproduced, the second picture 
being different from a leading picture of the leading sec- 
ond GOP. The above-described objective is achieved by 
this. 

[0040] The reproduction control information may 
include information representing the number of pictures 
from the leading picture of the leading second GOP to 
the second picture. 

[0041] The reproduction control information may 
include information representing the time to reproduce 
the leading picture of the leading second GOP and the 
time to reproduce the second picture of the leading sec- 
ond GOP. 

[0042] The reproduction control information may 
include timing information representing the timing to 
start decoding the leading second GOP so that repro- 
duction completion time of the first picture matches the 
reproduction start time of the second picture. 
[0043] The timing information may represent the 
timing to start decoding the leading second GOP when 
a picture which is not necessary for decoding pictures 
from the leading picture of the leading second GOP to 
the second picture is not decoded. 
[0044] The picture which is not necessary may be a 
B picture. 

[0045] An optical disk recording apparatus accord- 
ing to the present invention includes a generation sec- 
tion for generating reproduction control information; and 
a recording section for recording the reproduction con- 
trol information on an optical disk having, recorded ther- 
eon, a first video stream including a plurality of first 
GOPs and a second video stream including a plurality of 
second GOPs. The reproduction control information 
represents that after a first picture included in a final first 
GOP among the plurality of first GOPs included in the 
first video stream is reproduced, a second picture 
included in a leading second GOP among the plurality 
of second GOPs included in the second video stream is 
reproduced, the second picture being different from a 
leading picture of the leading second GOP. The above- 
described objective is achieved by this. 
[0046] An optical disk recording apparatus accord- 
ing to the present invention includes an editing section 
for editing a first video stream including a plurality of first 
GOPs and a second video stream including a plurality of 
second GOPs so that at least one picture unnecessary 
for reproduction is deleted in accordance with the repro- 
duction control information; and a recording section for 
recording the edited first video stream and the edited 
second video stream on an optical disk. The reproduc- 
tion control information represents that after a first pic- 
ture included in a final first GOP among the plurality of 
first GOPs included in the first video stream is repro- 
duced, a second picture included in a leading second 
GOP among the plurality of second GOPs included in 
the second video stream is reproduced, the second pic- 
ture being different from a leading picture of the leading 



second GOP. The above-described objective is 
achieved by this. 

[0047] The at least one picture unnecessary for 
reproduction may include a picture, of the first video 

5 stream, after the first picture, and a picture, of the sec- 
ond video stream, before the second picture. 
[0048] The at least one picture unnecessary for 
reproduction may further include a picture which is not 
necessary for decoding pictures from the leading pic- 

w ture of the leading second GOP in the second video 
stream until the second picture. 

[0049] The at least one picture unnecessary for 
reproduction may be a B picture. 

[0050] The recording section may record the edited 
15 first video stream and the edited second video stream in 
continuous regions of the optical disk. 
[0051] The recording section may record the repro- 
duction control information on the optical disk. 
[0052] The recording section may record the repro- 
20 duction control information on a medium other than the 
optical disk. 

BRIEF DESCRIPTION OF DRAWINGS 



25 [0053] 

Figure 1 is a block diagram of a 720P/480P hierar- 
chical recording apparatus in one example accord- 
ing to the present invention. 

30 Figure 2 is a block diagram of a 480i/480P/720P 

(60) reproduction apparatus in one example 
according to the present invention. 
Figure 3 is a block diagram of a 480P/720P (24/60) 
reproduction apparatus in one example according 

35 to the present invention. 

Figure 4 is a block diagram of a reproduction appa- 
ratus (720P output) of horizontal direction synthesis 
system in one example according to the present 
invention. 

40 Figure 5 is a block diagram of a three hierarchical 

layer optical disk recording apparatus in one exam- 
ple according to the present invention. 
Figure 6 is a block diagram of a recording and 
reproduction apparatus of a frame-based reproduc- 

45 tion control system in one example according to the 

present invention. 

Figure 7 shows a process for recording steams by a 
recording apparatus of a reproduction control infor- 
mation recording system in one example according 

50 to the present invention. 

Figure 8 is a diagram comparing reproduction of an 
optical disk in one example according to the 
present invention by an existing reproduction appa- 
ratus and by a reproduction apparatus according to 

55 the present invention. 

Figure 9 is a graph illustrating the relationship 
between the recording time period and the capacity 
of an optical disk in one example according to the 
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present invention. 

Figure 10 is a block diagram of a hierarchical repro- 
duction apparatus of a 480P reproduction mode in 
one example according to the present invention. 
Figure 11 is a table showing a data structure of 5 
reproduction control information in one example 
according to the present invention. 
Figure 12 is a diagram illustrating a process for 
recording a plurality of streams by a recording 
apparatus in one example according to the present 10 
invention and a process for reproducing the plurality 
of streams by a reproduction apparatus in one 
example according to the present invention. 
Figure 13 is a flowchart illustrating reproduction 
control of two streams performed based on the 15 
reproduction control information by a reproduction 
apparatus in one example according to the present 
invention. 

Figure 14 is a table showing a data structure of the 
reproduction control information when the time 20 
stamps of the streams are continuous in one exam- 
ple according to the present invention. 
Figure 15 is a diagram illustrating a process for 
recording and reproduction performed by a record- 
ing and reproduction apparatus in one example 25 
according to the present invention. 
Figure 16 is a flowchart illustrating a process for 
editing and generation of reproduction control infor- 
mation performed by a recording apparatus in one 
example according to the present invention. 30 
Figure 17 is a table showing a data structure of a 
picture identifier, representing information including 
resolution, of management information data in one 
example according to the present invention. 
Figure 18 is a block diagram showing an MPEG 35 
decoder of a reproduction apparatus of a different 
system in one example according to the present 
invention; 

Figure 19 is a diagram illustrating the principle of 
multiple angle picture data division multiplex 40 
recording system in one example according to the 
present invention. 

Figure 20 is a diagram illustrating a method for 
recording horizontal and vertical interpolation infor- 
mation in interleave blocks after being divided in 45 
one example according to the present invention. 
Figure 21 is a view illustrating the principle of an 
MADM system for dividing a signal into two in a hor- 
izontal, direction in one example according to the 
present invention. 50 
Figure 22 is a diagram illustrating picture synthesis 
control performed by a reproduction apparatus in 
one example according to the present invention. 
Figure 23 is a diagram showing signal arrangement 
for outputting a progressive signal, an NTSC signal 55 
and a HDTV signal in one example according to the 
present invention. 

Figure 24 is a timing diagram of reproduction of pro- 



gressive, 3D and wide signals with respect to the 
data amount in buffer in one example according to 
the present invention. 

Figure 25 is a block diagram of a reproduction 
apparatus in one example according to the present 
invention in an interlace video signal output mode. 
Figure 26 is a flowchart illustrating a method for 
performing AV synchronization of a first decoder 
and a second decoder in one example according to 
the present invention. 

Figure 27 is a flowchart illustrating a method for 
controlling two buffer sections in one example 
according to the present invention. 
Figure 28 is a timing diagram showing a data 
stream which is reproduced and output after proc- 
essed with buffering and decoding by the decoder 
in one example according to the present invention. 
Figure 29 is a flowchart showing a detailed process 
for reproducing a program chain group by a system 
control section M1-9 in one example according to 
the present invention. 

Figure 30 is a block diagram showing a structure of 
a part of an AV synchronization control 12-10, the 
part performing AV synchronization, in one exam- 
ple according to the present invention. 
Figure 31 is a block diagram of a data decoding 
section in one example according to the present 
invention. 

Figure 32 shows a signal format of a picture identi- 
fier in one example according to the present inven- 
tion. 

Figure 33 is a flowchart illustrating a process for 
STC switching for seamless connection in one 
example according to the present invention. 
Figure 34 is a view illustrating processing of a hori- 
zontal filter circuit in one example according to the 
present invention. 

Figure 35 is a block diagram showing a structure of 
an optical disk reproduction apparatus in one 
example according to the present invention. 
Figure 36 is a structural view of a video decoder in 
one example according to the present invention. 
Figure 37 shows a data structure of an optical disk 
in one example according to the present invention. 
Figure 38 is a timing diagram of video reproduction 
in one example according to the present invention. 
Figure 39 is a block diagram showing a structure of 
an optical disk reproduction apparatus in one 
example according to the present invention. 
Figure 40 is a structural view of an audio decoder in 
one example according to the present invention. 
Figure 41 shows a data structure of an optical disk 
in one example according to the present invention. 
Figure 42 is a timing diagram of audio and video 
reproduction in one example according to the 
present invention. 

Figure 43 shows an optical disk reproduction appa- 
ratus in one example according to the present 
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invention. 

Figure 44 is a structural view of a video decoder in 
one example according to the present invention. 
Figure 45 is a timing diagram of video reproduction 
in one example according to the present invention. 5 
Figure 46 is a block diagram showing a structure of 
an optical disk reproduction apparatus in one 
example according to the present invention. 
Figure 47 is a structural view of a video decoder in 
one example according to the present invention. 10 
Figure 48 is a structural view of a video decoder in 
one example according to the present invention. 
Figure 49 is a structural view of a video decoder in 
one example according to the present invention. 
Figure 50 is a block diagram showing a structure of 15 
an optical disk reproduction apparatus in one 
example according to the present invention. 
Figure 51 is a structural view of an audio decoder in 
one example according to the present invention. 
Figure 52 shows a data structure of an optical disk 20 
in one example according to the present invention. 
Figure 53 is a timing diagram of audio and video 
reproduction in one example according to the 
present invention. 

Figure 54 is a timing diagram of operation frequen- 25 
cies of audio reproduction in one example accord- 
ing to the present invention. 

Figure 55 is a timing diagram of operation frequen- 
cies of audio reproduction in one example accord- 
ing to the present invention. 30 
Figure 56 is a diagram illustrating a flow of a stream 
in a reproduction apparatus in one example accord- 
ing to the present invention. 

Figure 57 is a flowchart illustrating a process for 
MPEG encoding, editing/reproduction control infor- 35 
mation generation, and reproduction control per- 
formed by a recording and reproduction apparatus 
in one example according to the present invention. 
Figure 58 is a block diagram of a recording and 
reproduction apparatus of a frame-based reproduc- 40 
tion control system in one example according to the 
present invention. 

Figure 59 is a diagram illustrating a process for 
deleting an unnecessary frame in one example 
according to the present invention. 45 
Figure 60 is a block diagram of a reproduction 
apparatus of a mutual authentication system and a 
TV monitor in one example according to the present 
invention. 

50 

BEST MODE FOR CARRYING OUT THE INVENTION 

[0054] Hereinafter, the present invention will be 
described by way of examples with reference to draw- 
ings. 55 



(Example 1) 

(720P/480P hierarchical recording and reproduction 
system) 

[0055] With reference to Figure 1, a specific hierar- 
chical recording apparatus for handling two hierarchical 
layers of 720P and 480P will be described. Later, a 
method for recording a HDTV signal in a hierarchical 
manner in the state where the HDTV signal is divided 
into a plurality of signals will be described, with refer- 
ence to Figure 20. 

[0056] In the case of a movie signal, specifically, an 
original 720P video signal of 60 frames per second, the 
signal is input and then has extra frames deleted by a 3- 
2 pull-down section 746. As a result, a 720P (24P) sig- 
nal 703 of 24 frames per second is obtained. In the case 
of a normal 60P video signal, the 3-2 pull-down section 
is bypassed. Herein, 60P refers to 60 frames per sec- 
ond. The 720P video signal 703 having 1280 x 720 pix- 
els is processed by a 720P/480P down-converter 704 
as follows. First, the number of vertical lines is reduced 
to 720 x 2/3 = 480 by a vertical filter 705. Then, the 
number of pixels is reduced to 1280 x 9/1 6 = 720 pixels 
by a horizontal filter 706. Thus, the 720P video signal 
703 is converted into a 480P video signal 707 having 
720 x 480 pixels. Such a low resolution 480P video sig- 
nal is encoded by an MPEG encoder 708 for 480P into 
a compression MPEG signal. Then, the compression 
MPEG signal is decoded back into a 480P video signal 
710 by an MPEG decoder 709. This signal is enlarged 
to 3/2 times and 1 6/9 times respectively by a vertical fil- 
ter 712 and a horizontal filter 713 in a 480P/720P up- 
converter 711, and thus is converted into a 720P high 
resolution video signal 714. The original 720P video sig- 
nal 703 and the 720P video signal 714 obtained by 
MPEG encoding and decoding are differential-calcu- 
lated by a calculation circuit 715 in a differential signal 
processing 720, and thus differential information 716 is 
obtained. 

[0057] The differential information 716 is encoded 
by a second MPEG encoder 717 for 720P into a GOP- 
based video signal including an intraframe (i picture) 
and a differential frame (P or B). This signal is divided by 
multiplex means 719 into GOP-based second interleave 
blocks 718a and 718b including 1GOP to nGOP An 
MPEG stream of a basic signal encoded by the first 
MPEG encoder 708 for 480P in a basic signal process- 
ing section 721 is made into a480P GOP-based MPEG 
stream and then divided by the multiplex means 719 
into first interleave blocks 722a and 722b. The first inter- 
leave blocks 722a and 722b are interleaved into the 
second interleave blocks 718a and 718b; i.e., the first 
interleave blocks 722a and 722b and the second inter- 
leave blocks 718a and 718b are alternately arranged. 
The resultant signal is recorded on a disk 724 such as a 
DVD or the like by recording means 723. Also recorded 
at this point are a hierarchical recording identifier 725 
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indicating the start point and the termination point and 
specified interleave block reproduction prohibition infor- 
mation 726 for prohibiting the second interleave blocks 
718a and 718b including the differential information 
from being reproduced by the conventional reproduction 5 
apparatus. The identifier and information are recorded 
in overall management information 224 and each of 
VOBs as shown in Figure 23. 

[0058] When the disk 724 is reproduced by the 
existing reproduction apparatus based on the DVD 10 
standards as shown in Figure 8, the interleave blocks 
722a and 722b are regarded as a first angle and repro- 
duced. The reproduction signal is decoded by MPEG 
data 727, and thus NTSC or 480P (24 frames) video 
signal is reproduced. As shown in Figure 23, the speci- 15 
fied interleave block reproduction prohibition informa- 
tion 726 for prohibiting specified interleave blocks 
including the differential information from being repro- 
duced, for example, an angle switching prohibition flag 
is recorded. Accordingly, even if the user inadvertently 20 
operates the reproduction apparatus, reproduction of a 
second angle, i.e., the second interleave unit is prohib- 
ited. In other words, the differential information for 720P 
is automatically prevented from being reproduced by 
the existing DVD reproduction apparatus. When the dif- 25 
ferential information for 720P is reproduced in error, 
malfunction occurs since this signal cannot be normally 
reproduced by a first MPEG decoder for 480i of the 
existing reproduction apparatus. This type of trouble is 
avoided by the present invention. In this case, informa- 30 
tion on connection to the second interleave blocks may 
be intentionally excluded from the management infor- 
mation 224, which is referred to as the navigation infor- 
mation in the DVD standards. 

[0059] The above-described effect is also useful 35 
when a 720P signal itself is recorded in the second 
interleave blocks. In this case, the 720P signal is directly 
input to the MPEG encoder 717 as shown by the arrow 
indicated with "*" in Figure 1. 

[0060] In this manner, when the disk 724 is repro- 40 
duced by the existing DVD reproduction apparatus, a 
video signal is reproduced at the quality equivalent to 
NTSC, which is obtained by reproducing an existing 
DVD disk; and furthermore erroneous reproduction of 
information which cannot be normally reproduced by 45 
the existing DVD reproduction apparatus, such as a dif- 
ferential signal or a 720P signal, is prevented. Thus, 
bidirectional compatibility is realized. 
[0061] A 480P signal itself may be recorded in the 
second interleave blocks instead of the 720P signal. In 50 
this case, the first interleave blocks are reproduced and 
thus a 480i (NTSC) signal is output by the conventional 
reproduction apparatus. By a reproduction apparatus 
according to the present invention, a 480i signal from 
the first interleave blocks or a 480P signal from the sec- 55 
ond interleave blocks is reproduced, or both of them can 
be reproduced. 

[0062] When a reproduction apparatus according to 



the present invention is used, a basic signal is repro- 
duced from the first interleave blocks 722a and 722b, 
which is referred to as the first angle in the DVD stand- 
ards. A differential signal and a 720P signal are repro- 
duced from the second interleave, blocks 718a and 
718b, which is referred to as the second angle in the 
DVD standards. From the first angle, a 480P video sig- 
nal 729 is output by an MPEG decoder 728 for 480P; 
and from the second angle, a 720P video signal 731 or 
a 720P signal as a differential signal is reproduced by 
an MPEG decoder 730 for 720P. These two video sig- 
nals having a different number of pixels are synthesized 
by a synthesis section 732 or output as they are, and 
thus decoded into an original 720P video signal 733 to 
be output. 

[0063] In this manner, when the hierarchical record- 
ing disk 724 is reproduced by the reproduction appara- 
tus according to the present invention, a 720P video 
signal is output. Thus, a HDTV signal such as a 720P 
signal can be recorded while compatibility with the con- 
ventional reproduction apparatus is maintained. 
[0064] When a 480P signal itself is recorded in the 
second interleave blocks, a 480 P signal having a den- 
sity twice as high as that of an NTSC signal is repro- 
duced. 

[0065] With reference to Figure 3, a more specific 
operation of the reproduction described with reference 
to Figure 8 will be described. The blocks which have 
already been described will not be described again. 
[0066] The disk 724 has a basic signal and a differ- 
ential signal recorded thereon alternately after being 
divided on an nGOP-by-nGOP basis by the multiplex 
means 719 shown in Figure 1 and interleaved. This sig- 
nal is divided into the first interleave block 722a and the 
second interleave block 718a, i.e., the basic signal and 
the differential signal, by a division section 734 of the 
reproduction apparatus shown in Figure 3. Then, the 
basic signal and the differential signal are stored in a 
first buffer memory 735 and a second buffer memory 
736 respectively. Then, respective time information is 
extracted from a time information extraction section 
793. A VTS synchronization section 780 sets first basic 
time information and second basic time information in 
the first decoder 728 and the second decoder 730 so 
that the two signals are in synchronization with each 
other. Thus, output signals from the two decoders are 
synchronized. In this case, when the hierarchical 
recording identifier 725 is detected, an identification 
information processing section 745 recognizes that a 
first reproduction signal which is the decoded signal of 
the first stream is a basic signal having a smaller 
number of pixels and that a second reproduction signal 
which is the decoded signal of the second stream is a 
differential signal having a larger number of pixels and 
having the differential information from the basic signal. 
Thus, the identification information processing section 
745 give the synthesis section 732 an instruction 
regarding an up-converter 738 and an instruction of 
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addition. 

[0067] The MPEG decoder 728 for 480P and the 
MPEG decoder 730 respectively decode the signals 
into a 480P (24) signal and a 720 (24 frames) signal. 
The decoded signals have 24 frames/sec. or 30 
frames/sec. The signals are processed by 2-3 conver- 
sion sections 737a and 737b so as to output the same 
frame twice, and thus a 480P signal 729 of 60 
frames/sec. and a 720P signal 731 having the differen- 
tial information are obtained. The 480P signal 729 is up- 
converted into a 720P signal 739 by the 480P/720P up- 
converter 738 and added to the 720P signal 731 having 
the differential information by an addition section 740, 
and thus the original 720P video signal 733 is obtained. 
The addition section 740 calculates, for example, as 
shown in the figure. Where the pixels of the respective 
signals are a and b, (a+b)/2 is performed to obtain the 
original 720P video signal 733. The calculation per- 
formed by the synthesis section 732 may be different 
from (a+b)/2. 

[0068] In this case, the MPEG decoding signals 
may be kept to have 24 frames/sec. without being con- 
verted by the 2-3 conversion sections 737a and 737b to 
have 60 frames/sec. and after synthesis, converted to 
have 60 frames/sec. by a 2-3 conversion section 741 . In 
such a case, the amount of data of the video signal is 
advantageously reduced to half, and the processing 
ability of the digital processing circuit can be reduced to 
half. 

[0069] With reference to Figures 1 and 3, the 
method for recording and reproducing a 720P signal of 
24 frames/sec. such as a movie signal in a hierarchical 
manner has been described. This method has signifi- 
cant advantages. The HDTV format includes the 1080i 
system and the 720P system. As shown in Figure 9, a 
1080i signal (24 frames) used for movies can be 
recorded only for 90 minutes since a two- layer DVD has 
a capacity of 8.5 GB as indicated by curve 742a. 
[0070] By contrast, a 720P signal (24 frames) can 
be recorded for 150 minutes as indicated by curve 
742b. A 480P signal (60 frames) can be recorded for 
150 minutes as indicated by curve 742c. Disks for mov- 
ies are considered to be meaningless unless each has 
a recording capacity of 120 minutes or more. The 720P 
(24)/480P hierarchical recording disk has an effect that 
an HDTV movie title can be accommodated in one DVD. 
[0071] In the example shown in Figure 3, a 480P 
signal having the basic information of a 720P signal is 
recorded in the first interleave blocks, and the differen- 
tial information between the 720P and 480P signals is 
recorded in the second interleave blocks. For a disk 
having a 720P signal as it is in the second interleave 
blocks is reproduced, the output from the second 
decoder 730 can be output as it is as shown by arrow 
indicated with "*" in Figure 3. The determination to do 
this is made by the identification information processing 
section 743 based on an identifier. In this case also, an 
effect equivalent to complete compatibility is obtained. 



This system has a lower recording efficiency but has the 
effect of simplifying the processing circuit for recording 
and reproduction and the effect of the complete compat- 
ibility. 

5 [0072] With reference to Figure 60, an example in 
which a decoder is mounted on a TV monitor 798 will be 
described. The basic operation is the same as that 
described with reference to Figure 3, and only the differ- 
ent parts will be described. In a reproduction apparatus 

10 743a, a signal before decoding is encrypted by an 
encryption encoder 795 using an encryption key 799a 
and sent from a communication interface section 796a 
through a network 798 to a communication interface 
section 796b in the TV monitor 798. Before this opera- 

15 tion, mutual authentication sections 794a and 794b 
communicate to each other to authenticate each other. 
This operation may be referred to as a handshake. In 
the case where mutual authentication is confirmed and 
thus both sides determine that the other side is proper, 

20 the mutual authentication sections 794a and 794b pro- 
vide the encryption encoder 795 and an encryption 
decoder 797 with the encryption keys 799a and 799b 
and also permit the communication interface 796a and 
796b to communicate. Thus, encryption data is sent 

25 and received, and the keys of the encryption data are 
unlocked. Therefore, the first stream and the second 
stream are respectively sent to the first decoder 728 
and the second decoder 730. The determination to con- 
duct this processing is made by the identification infor- 

30 mation processing section 745 based on an identifier 
744 which is separately sent. When the first stream is of 
a 480P signal and the second stream is of a 720P differ- 
ential signal as described above, up-conversion and 
synthesis calculation are performed, and a 720P signal 

35 is output to a TV monitor 798a. When an identifier indi- 
cating that the second stream is of a 480P differential 
signal is received, the two streams are synthesized to 
output a 480P signal. When an identifier indicating that 
the streams are of a 3D signal, a 3D signal is output and 

40 displayed on the TV monitor 798a. The 3D signal is 
obtained by time-based synthesis, where the first 
stream is set for the left eye and the second stream is 
set for the right eye. 

[0073] According to this system, even when two 
45 streams are encrypted, the streams are processed, for 
example, synthesized by the TV monitor using the iden- 
tifier 744. Thus, the original picture can be obtained 
without violating the copyright protection secured by the 
encryption. 

50 [0074] Next, with reference to Figure 10, an opera- 
tion for reproducing a disk 724a having a 480P (60 
frames/sec.) recorded thereon by the reproduction 
apparatus according to the present invention will be 
described. Common parts as those in Figure 3 will not 

55 be described. 
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(Sum and difference system - Figure 19) 

[0075] With reference to Figure 19, the concept of 
the sum and difference system will be described. This 
system is referred to as multiple angle video data divi- 
sion multiplex system (MADM) since a video signal is 
divided into vertical or horizontal high frequency and low 
frequency components and recorded in each of multiple 
angles. As shown in Figure 19, a signal is divided into a 
basic signal (sum signal) and a sub signal (differential 
signal) by a sum calculation section 141 and a differen- 
tial calculation section 143. The resultant signals are 
MP EG -encoded and then alternately recorded as inter- 
leave blocks in units of 1 GOP. At this point, the amount 
of the information can be reduced by 20% by performing 
a 3-2 conversion of the basic signal and the sub signal 
in synchronization with each other. It is efficient to use, 
as the basic signal, "IBBPBBPBBPBBPBB" which is 
shown as a main GOP structure 244 used for the ordi- 
nary MPEG encoding. In this structure, an I frame 246, 
B frames 248 and P frames 247 are alternately 
arranged. In the case of the differential signal, however, 
experiments have shown that it is efficient to have a 
structure including only I frames 246 and P frames 247 
due to the profile pattern, for example, "IPPPPPP- 
PIPPPPPPP" shown as a sub GOP structure 245. The 
efficiency is improved by changing the setting for the 
sub GOP structure. 

[0076] Figure 19 shows an example in which a 
480P video signal is divided into two in a vertical direc- 
tion. Figure 21 (described below) shows an example in 
which a 480P video signal is divided into two in a hori- 
zontal direction. In an alternative manner, a 60-frame 
480P signal may be divided by frame division means 
into 30 odd frames and 30 even frames. In this case, the 
respective 30P signals are converted into two 60-field 
interlace signals, and each of the signals are MPEG- 
encoded and recorded in the MADM system. Such 
encoding is performed in a progressive manner, and 
therefore encoding efficiency is improved as in the case 
of the movie. Thus, the recordable time period of the 
same disk is extended. 

[0077] When such a signal is reproduced by a non- 
MADM reproduction apparatus, a 30P (one-channel) 
525 interlace signal is reproduced in a first channel. 
Such a signal lacks necessary frames and is distorted. 
[0078] When such a signal is reproduced by an 
MADM reproduction apparatus, a 30P signal is repro- 
duced as a basic signal and another 30P signal is repro- 
duced as a sub signal. These two 30-frame signals are 
synthesized into a 60-frame normal 480P signal by 
frame synthesis means including a frame buffer, and 
then output. 

[0079] When a line doubler is added to an output 
section for the 480P signal, a 1050P video signal is 
obtained. 

[0080] When a 525 interlace signal is input to a sum 
signal section of the synthesis section of the MADM 



reproduction apparatus and the value of 0 is input to a 
differential signal section of the synthesis section, a 
480P video signal is obtained. Such a manner of input 
has the same effect as the line doubler. This method 

5 allows even a 525 interlace signal to be output as a 
480P signal. Accordingly, all types of pictures can be 
viewed by simply connecting one cable to a progressive 
input terminal of the MADM reproduction apparatus. 
[0081] In Figure 19, 1/2(A+B) and 1/2(A-B) are 

w used as expressions for calculation for a two-tap, filter. 
The division frequency corresponds to about 300 scan- 
ning lines. 

[0082] As shown in Figure 19, the disk 724a having 
a 480P signal in the state of being divided into two sig- 

15 nals by sum and differential calculations and recorded in 
two block groups, i.e., the first interleave blocks and the 
second interleave blocks is reproduced. The signal is 
divided into a 480i signal as a basic signal and a 480i 
signal as a differential signal by the division section 734. 

20 The signals are respectively decoded by the MPEG 
decoders 728 and the MPEG decoder 730 to obtain a 
480i signal 729a and a 480i differential signal 731a. The 
addition section 740 performs the calculation of (a+b)/2 
to synthesize the two 480i signals. Thus, a 480P (60 

25 frames) synthesis signal 733a is output. 

[0083] The disk 724 includes 480i/480P/720P iden- 
tification information 744 (Figure 17) recorded in a toe 
section or the like thereof. The 480i/480P/720P identifi- 
cation information 744 includes three signals in the case 

30 of the 480i signal, the 480P signal and the 720P signal, 
and also shows which resolution of differential signals 
are recorded. The identification information processing 
section 743 processes this information to determine in 
which sector address of the disk main data (main signal) 

35 and sub data (differential signal) of the hierarchical data 
are recorded. The identification information processing 
section 743 then sends information on the start point 
and the like to the synthesis section 732. The synthesis 
section 732 performs a synthesis calculation of the main 

40 data and the sub data from the start point of the 480P 
signal and outputs a 480P (60 frames/sec.) signal. 
[0084] As shown in Figure 17, row of Vts=6, it is 
recorded on the disk that at the start point of the 720P 
signal, 720P-main is the first interleave block, and 

45 720P-sub is the second interleave block. The identifica- 
tion information processing section 743 identifies this 
information. The calculation section 740 performs a 
synthesis calculation of the 720P signal, for example, 
(a+b)/2 from the start time stamp of the 720P signal, 

50 using the time stamps of the main signal and the differ- 
ential signal from the MPEG decoders 728 and 730, and 
then outputs a 720P signal. 

[0085] When a 480P identifier is recorded as the 
identification information 744 (Figure 17), the identifica- 
55 tion information processing section 745 sends a 480i 
signal decoding instruction to the MPEG decoder 730 
and causes the MPEG decoder 730 to decode the 480i 
signal. Then, a differential signal 731 a of the 480i signal 
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is decoded and synthesized by the synthesis section 
732. Thus, a 480P (60 frames/sec.) is output. 
[0086] In this manner, the MPEG decoder 730 per- 
forms the 480i processing (480P-30 frames/sec.) or the 
720P processing in accordance with the identification 
information. Thus, the main signal and the differential 
signal of the 480i signal, and the main signal and the 
sub signal of the 720P signal, can be decoded by two 
MPEG decoders in total. This has the effect of simplify- 
ing the structure of the apparatus. 
[0087] The 480P reproduction mode shown in Fig- 
ure 10 does not use the 480P/720P up-converter 738 in 
the synthesis section 732, but allows the decoded 480P 
(60) signal to be up-converted to a 720P signal by the 
480P to 720P up-converter 738 and thus displayed on a 
HD video projector for 720P or the like. Thus, the scan- 
ning lines are advantageously more unlikely to be 
viewed. In this case, one 480P to 720P up-converter 
738 can be used for 720P signal synthesis and 480P to 
720P up-conversion. Thus, the 480P signal can advan- 
tageously be up-converted into a 720P signal without 
adding any element. 

(720P/480P/480i three hierarchical layer recording 
apparatus) 

[0088] With reference to Figure 5, a structure and 
an operation of the 720P (60 frames/sec. )-type three 
hierarchical layer recording apparatus will be described. 
The structure and the operation are substantially the 
same as those in Figure 1, and only the different parts 
will be described. First, an input signal is a 720P signal 
of 60 frames/sec. Accordingly, after being 480P down- 
converted, the input signal is a 480P signal of 60 
frames/sec. This signal is input to a basic signal 
processing section 721a and processed by a division 
section 747. Where the pixel data of the n'th line is a and 
the pixel data of the (n+1 )th line is b, the division section 
747 uses the calculation result of (a+b)/2 for the m'th 
line of a 480i video signal 748a and the calculation 
result of (a-b)/2 for the m'th line of a 480i video signal 
748b, and thus obtains a main signal and a sub signal 
of an NTSC signal. These signals are respectively 
encoded by the MPEG encoders 708a and 708b, and 
decoded by the MPEG decoders 709a and 709b into 
decoded signals 749a and 749b. The signals are syn- 
thesized by a synthesis section 748 into a 480P signal 
710. The 480P signal is up-converted into a 720P signal 
714, and differential information is obtained. The differ- 
ential information is MPEG-encoded to obtain data as 
third interleave block data 718a and 718b. This proce- 
dure is substantially the same as that in Figure 1 except 
that the frame rate is 60 frames/sec. instead of 24 
frames/sec. 

[0089] The 480i MPEG stream is divided by multi- 
plex means 719a into interleave blocks on an nGOP 
basis. The nGOPs are interleaved in the order from a 
480i signal (first interleave blocks 722a of a basic sig- 



nal), then a 480i signal (second interleave blocks 750a 
of a differential signal), and then a 720P (third interleave 
blocks 71 8a of a differential signal), and recorded on the 
disk 724 such as a DVD. 

5 [0090] In this case, the multiplexed signals are 
modulated by an 8VSB, QAM or OFDM modulation sec- 
tion 751 and transmitted from a transmission section 
752. Thus, hierarchical broadcasting can be performed. 
The signals may be multiplexed by time division based 

10 on a time domain defined by the broadcasting instead of 
based on a GOP. 

[0091] In this manner, a 480i/480P (60)/720P three 
hierarchical layer disk or hierarchical broadcasting is 
realized. 

15 [0092] With reference to Figure 2, an operation for 
reproducing the disk 724a will be described. Since iden- 
tical elements with those in Figure 3 are included, the 
identical elements will not be described. A signal repro- 
duced from the disk 724a or received by a receiving 

20 section 753 and demodulated by a demodulation sec- 
tion 754 is divided by the division section 734 into three 
streams based on the above-mentioned interleave 
blocks. The three streams are decoded by the three 
MPEG decoders 728a, 728b and 730 through buffers 

25 735a, 735b and 736. Then, three signals, i.e., the 480i 
signal 749a, the 480i differential signal 749b, and the 
720P differential signal 731 are obtained as a result of 
demodulation. By subjecting the 480i basic signal 749a 
and the 480i differential signal 749b to the calculations 

30 of (a+b) and (a-b) by a synthesis section 732, a 480P 
(60 frames/sec.) video signal 729 can be obtained. This 
signal and the above-mentioned 720P differential signal 
731 are synthesized into a 720P output 733a. The pro- 
cedure is described above and will not be repeated. 

35 [0093] In this manner, three types of outputs of 480i 
output 749a, 480P output 729 and 720P output 733a 
having different resolutions can be obtained from the 
disk 724a. The user can select the output by the grade 
of the monitor reproduction apparatus. That is, the 480i 

40 (NTSC) grade output is obtained by the existing repro- 
duction apparatus, the 480P (60 frames/sec.) is 
obtained by the reproduction apparatus for 480 P 
according to the present invention, and the 720P (60 
frames/sec.) is obtained by the reproduction apparatus 

45 for 720P according to the present invention. Thus, the 
complete compatibility is realized. 
[0094] In Figure 2, when the identification informa- 
tion processing section 745 detects a high resolution 
identifier, the rotation rate of the motor is raised through 

so a system control section 21 and a rotation control circuit 
35. A high resolution signal can be reproduced by rais- 
ing the rotation rate in accordance with the identifier. 
The rotation rate is raised to 1X for reproduction of a 
standard picture, 2X for 480P and 720P (24P), and 3X 

55 to 4X for 720 P (60 P). The effect of power saving is pro- 
vided. Then an NTSC grade signal is reproduced, the 
system control section 21 stops or operates, at a low 
rate, clocks of the 720P MPEG decoder 730, the 480i 
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MPEG decoder 728b, and the synthesis section 732, 
which are not necessary. Thus, power consumption can 
be significantly reduced. Then an ATPS 84 of the audio 
time stamp of audio data is received by an AV synchro- 
nization control section 158 and a video presentation 
time stamp VPTS for each of the MPEG decoders is 
created based on the time information and set in regis- 
ters 39a, 39b and 39c of the decoders, frames can be 
synchronized for reproduction from the decoders. In 
order to synchronize the vertical blanking, a decoder 
synchronization section 794 simultaneously resets the 
horizontal and vertical synchronization of the decoders. 
The pictures from the decoders can be synchronized on 
a dot-by-dot basis. Specific synchronization methods of 
audio and video signals will be described later. 
[0095] From the disk 724a, a first resolution identi- 
fier indicating a low resolution of NTSC signals or the 
like of the picture of the first stream and a second reso- 
lution identifier indicating a high resolution of 720P sig- 
nals or the like of the second and third streams are 
reproduced. The system control section 21 determines 
by calculation which processing is to be performed by 
the up-converter 738 in the synthesis section 732 
among 480P to 720P, 480P to 1080i, 480P to 1080P, 
and 720P to 1080P, and indicates the result to the syn- 
thesis section 732. In actuality, various first resolution 
identifiers exist such as, for example, 704 x 480 and 720 
x 480. This has an effect that the up-converter operates 
at the optimum ratio. Needless to say, a simple system 
structure in which an identifier indicating the ratio of the 
up-converter is recorded and reproduced can be 
adopted. 

[0096] The reproduction apparatus 743a in Figure 2 
can output three resolutions of signals (i.e., 480i 
(NTSC) signal of the first stream, 480P (60P) signal 729 
of the first and second streams, and 720P (60P) signal 
733a of the first, second and third streams) simultane- 
ously or at different timing. Such a reproduction appara- 
tus can be used with monitors having various 
resolutions. 

[0097] Especially, since a 480P signal 729 can be 
converted into a 720P signal by the up-converter 738 of 
the synthesis section 732, the 720P signal obtained as 
a result of conversion of the 480P signal can be 
obtained without adding any circuit. 
[0098] In the case where a receiving section 753 
and a demodulation section 754 are added to the hier- 
archical reproduction apparatus, a receiving apparatus 
for receiving a hierarchical signal such as a TV signal, 
demodulating the signal and outputting three resolu- 
tions of video signals can be provided. 

(Wide 480P) 

[0099] With reference to Figure 21 , a concept of the 
MADM system in which a signal is divided in a horizon- 
tal direction is shown. The signal can be converted by a 
3-2 conversion section 174 into a 1440 x 480i interlace 



signal. The signal is divided into two in a horizontal 
direction by a horizontal filtering section 206a. The prin- 
ciple of filtering is shown in parts (a) and (b) of Figure 
34. As shown in part (b), 1440 dots are divided into odd 

5 dots 263a and 263b, and even dots 264a and 264b. 
Where the odd dots are labeled as Xn and the even dots 
are labeled as Yn, a sum signal is obtained by the cal- 
culation of X+Y and a differential signal is obtained by 
the calculation of X-Y. As a result, two 480P or 525i sig- 

10 nals, each of 720x480, are obtained as shown in part 
(b) of Figure 34. 

[0100] Returning to Figure 21, the number of hori- 
zontal dots of such a horizontal sum signal is reduced to 
720. Since the signal is passed through the horizontal 

15 filter, however, aliasing distortion is as low as that of an 
NTSC signal. A conventional reproduction apparatus 
reproduces only the sum signal and accordingly pro- 
vides a DVD picture of exactly the same quality. The dif- 
ferential signal represents only a profile formed of line- 

20 drawing. However, since the difference signal is 
restricted by a second video signal output restriction 
information provision section 179 (Figure 60) so as not 
to be reproduced by an ordinary reproduction appara- 
tus, no problem occurs. The sum signal and the differ- 

25 ential signal are respectively encoded into MPEG 
streams by a first encoder 3a and a second encoder 3b, 
and subjected to interleaving in units of an interleave 
block of 1 GOP or more and MADM-multiplexed. 
[0101] In the case of the movie, 3-2 conversion is 

30 performed by the 3-2 conversion 174 section and 
MADM-recorded as an MPEG signal together with the 
3-2 conversion information 174. 

[0102] In the case of the movie, 24 frames are 
reproduced in one second. Accordingly, a 1440 x 480P 

35 progressive picture is reproduced based on two inter- 
lace signals by a 2X reproduction apparatus. The scope 
size of the movie is 2.35:1 . The format of 1 440x480P is 
suitable for the scope size of 2.35:1 in terms of the 
aspect ratio. Thus, a wide screen 480P is effectively 

40 reproduced. 

[0103] A wide 480i hierarchical disk 724b is 
described with reference to Figure 21 above. With refer- 
ence to Figure 4, an operation for reproducing the disk 
724b by a W-480i reproduction apparatus will be 

45 described. When the disk 724b has information 
recorded at 24 frames/sec, a W-480P basic signal 757a 
and a W480P differential signal 757b are obtained by 
decoding performed by field frame conversion sections 
756a and 756b. Each pixel is encoded by data obtained 

50 by (X+Y)/2 or (X-Y)/2. Accordingly, when a synthesis 
section 758 performs the calculation of 
(X+Y)/2+(X-Y)/2 , X, i.e., data of odd pixels is obtained 
by the decoding. When the synthesis section 758 per- 
forms the calculation of (X+Y)/2-(X-Y)/2 , Y, i.e., data of 

55 even pixels is obtained by the decoding. As a result, the 
number of pixels in the horizontal direction is doubled to 
1440 pixels. In this manner, a W480P video signal 759 
of 1440 x 480P pixels is obtained. The W480P video 
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signal 759 of 1440 x 480P pixels is converted by a 
W480P-720P conversion section 760 so as to have 
1280 pixels in the horizontal direction using a 8/9 hori- 
zontal filter 760a, and so as to have 720 pixels in the 
vertical direction using a 3/2 vertical filter 760b. As a 
result, a 720P digital output is obtained. Thus, use of a 
general 720P digital interface is advantageously 
allowed. 

(Detailed reproduction operation: Figure 25) 

[0104] With reference to Figure 25, an operation of 
a reproduction apparatus 65 according to the present 
invention will be described in detail. Figure 25 is a block 
diagram of a reproduction apparatus for reproducing a 
2X progressive or super-wide picture or 720P signal. A 
signal reproduced from an optical disk 1 is divided by a 
division section 68 into first interleave blocks 66 and 
second interleave blocks 67 each including a frame sig- 
nal of 1 GOP or more. Frame video signals 70a and 70b 
each of 30 frames/sec. obtained as a result of MPEG 
extension performed by an extension section 69 are 
respectively divided by field division sections 71a and 
71b into odd field signals 72a and 72b and even field 
signals 73a and 73b. Thus, 2ch NTSC interlace signals 
74a and 74b are output. The wide screen shown in Fig- 
ure 20 will be described later. Referring to Figure 25, 
using the method described above, a 1440 x 960 pro- 
gressive picture 182a is divided in a horizontal direction 
using a sub band filter or wavelet conversion by a hori- 
zontal and vertical division section 194 in a picture divi- 
sion section 115. Thus, a 525 progressive video signal 
183 is obtained. This signal is divided into 525 interlace 
signals 184 and recorded as streams 188a and the like. 
[0105] Remaining interpolation information 185 is 
divided similarly into four streams 188c, 188d, 188e and 
188f and recorded as interleave blocks. The maximum 
transfer rate of each interleave block is 8 Mbps by the 
DVD standards. Accordingly, when the interpolation 
information is divided into four streams, the transfer rate 
is 32 Mbps. When the interpolation information is 
divided into six angles, the transfer rate is 48 Mbps. 
Thus, a 720P or 1050P HDTV video signal can be 
recorded. By the conventional reproduction apparatus, 
the stream 188a is reproduced to output an interlace 
video signal 184. Regarding the streams 188c, 188d, 
188e and 188f, output restriction information is 
recorded on an optical disk 187 by a picture processing 
restriction information generation section 179. There- 
fore, the interpolation information 185, such as differen- 
tial information or the like, which is not properly viewable 
is prevented from inadvertently being output. By divid- 
ing the signal in horizontal and vertical directions by the 
system shown in Figure 25, an optical disk compatible 
to both the HDTV and NTSC formats is advantageously 
realized. 

[0106] In Figure 25, the interlace signal obtained by 
the conversion performed by an interlace conversion 



section 175 is output to provide a scope screen 178. A 
480P progressive signal is similarly output on a scope 
screen 178. When a monitor for 720P is used, a 480P 
signal is converted into a 720P progressive signal by a 

5 480P/720P conversion section 176, and as a result, is 
output on a 1280 x 720 or 1440 x 720 letter box type 
720P screen 177 (the picture has 1280 x 480 or 1440 x 
480 pixels). Since the scope picture (2.35:1 ) has 1 1 28 x 
480 pixels, a picture having a size closer to the aspect 

10 ratio is obtained. Especially in the case of a movie, the 
signal is of 24 frames/sec. and so the progressive pic- 
ture is transferred at the rate of 4 Mbps. When a scope 
picture is recorded by the system according to the 
present invention of dividing the picture into two 

75 screens, the transfer rate is 8 Mbps. In such a case, 
about 2 hours of information can be recorded on a two- 
layer DVD. Accordingly, a 720P or 480P high quality pro- 
gressive picture signal for scope screen can be advan- 
tageously recorded on one DVD. On a conventional TV 

20 screen, the picture is displayed by an interlace output 
signal, needless to say. The present invention has an 
effect that a scope screen picture (2.35:1) of the movie 
can be output as a 480P or 720P signal. 

25 (High resolution recording identification information) 

[0107] Returning to Figure 1 , address information is 
output from an address circuit. A hierarchical recording 
identifier 725 including progressive/3D picture arrange- 
30 ment information is output from a progressive/3D pic- 
ture arrangement information output section 725a. 
These pieces of information are recorded on the optical 
disk by a recording circuit 723. The progressive/3D pic- 
ture arrangement information includes an identifier 
35 which indicates whether or not a progressive or 3D pic- 
ture is present on the optical disk, the hierarchical 
recording identifier 725 which indicates whether or not 
the signal is up-converted when being hierarchy- 
encoded, or a progressive/3D picture arrangement table 
40 14 shown in Figure 17. As shown in Figure 17, a TEX- 
TDT file 83 includes, for each VTS, 3D pictures for the 
right and left eyes and angle numbers and cell numbers 
in which the progressive signal is located. Since a PGC 
file of each VTS includes a start address and a termina- 
ls tion address of each cell, the start address and the ter- 
mination address of each cell are included in the 
progressive/3D picture arrangement information. Based 
on the arrangement information and identification infor- 
mation, the reproduction apparatus outputs a progres- 
50 sive picture or a 3D picture correctly as progressive 
outputs or R and L outputs. When ordinary pictures of 
different contents from each other are output as R and L 
outputs in error, the user will feel uncomfortable since 
the pictures for the right eye and the left eye are not 
55 related to each other. The progressive/3D picture 
arrangement information, the progressive/3D picture 
identifier, or the hierarchical recording identifier has an 
effect of avoiding the output of such unpleasant pic- 
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tures. As shown in Figure 3, when the hierarchical 
recording identifier 725 is reproduced, the control sec- 
tion sends an up-conversion instruction 786 to up-con- 
vert the a 480P signal into a 720P signal by the up- 
converter 738. Then, the synthesis of the 720P signal is 
performed. When the hierarchical recording identifier 
725 is not available, a 480P signal is output after per- 
forming synthesis calculation without using the up-con- 
verter 738 as shown in Figure 10. In this manner, stable 
picture synthesis can be performed simply by switching 
the connection in accordance with whether or not the 
identifier is available. 

[0108] With reference to Figure 23, a process for 
performing reproduction using a picture identifier 222 
will be described. 

[0109] From the optical disk, reproduction process 
control information 225 is first read from the manage- 
ment information 224. Since the information 225 
includes restriction information on VOB, a 0th VOB 
226a is only connected to a first VOB 226b having a 
main picture by an existing reproduction apparatus. 
Since the 0th VOB 226a is not connected to a second 
VOB 226c having an interpolation signal such as differ- 
ential information or the like, an ugly picture such as dif- 
ferential information is prevented from being output by 
the existing reproduction apparatus as described 
above. Each VOB of the main signal has a picture iden- 
tifier. Since the progressive identifiers and resolution 
identifier=00 (525) in the first VOB 226b and the second 
VOB 226c, a progressive signal having 525 scanning 
lines is reproduced from a progressive or HD reproduc- 
tion apparatus. 

[0110] In a picture identifier 222 of the next VOB 
226d, the progressive identifier=0 and the resolution 
identifier 219=10. This indicates that an interlace signal 
having 1050 scanning lines is output and that three 
VOBs 226e, 226f and 226g are interpolation informa- 
tion. Thus, an NTSC signal is output by a conventional 
reproduction apparatus, an interlace signal having 720 
horizontal pixels and 1050 vertical pixels is output by a 
progressive reproduction apparatus, and a full HDTV- 
format signal having 1050 scanning lines is output by a 
HD reproduction apparatus. As can be appreciated from 
this, various video signals can be recorded in an inter- 
leave manner and reproduced by the picture identifier 
222. The picture identifier 222 can be recorded in the 
management information 224. 

(2X clock and soft-decoding) 

[0111] In the block diagrams shown in Figures 3 
and 4, two MPEG decoders are used. In Figure 18, a 
first MPEG signal and a second MPEG signal are syn- 
thesized into one MPEG signal by a synthesis section 
36, and a 2X clock is generated by a 2X clock genera- 
tion section 37. The MPEG signal is doubled by a 2X 
clock-type MPEG decoder 16c, extended, and output as 
R and L video signals by a division section 38. In this 



manner, the circuit configuration can be simplified. This 
circuit configuration is realized simply by adding a 16 
MB SD-RAM to a memory 39 of the existing reproduc- 
tion apparatus, without significantly raising the cost. For 
5 soft-decoding, when a CPU has a 2X clock, one CPU 
realizes simultaneous decoding by time division. This 
will be described in a second example. 

(Simultaneous reproduction) 

10 

[0112] With reference to Figure 18, synchronous 
reproduction of two streams, which is important in 
decoding 3D picture data and progressive picture data 
will be described. First, it is necessary to adjust vertical 

15 and horizontal synchronization of two streams within a 
single line. In order to do this, a first MPEG decoder 16a 
and a second MPEG decoder 16b are started substan- 
tially simultaneously by a vertical/horizontal synchroni- 
zation control section 85c to synchronize the decoders 

20 16a and 16b. Then, it is necessary that the outputs from 
the two decoders should be a picture having an identical 
VPTS. This will be described with reference to the flow- 
chart in Figure 26 and Figure 18. In step 241a, the syn- 
chronization of a first decoder and a second decoder is 

25 cancelled. In step 241b, the decoders are synchronized 
with each other vertically and horizontally as described 
above. In step 241c, an APTS of an audio signal is read, 
and the APTS value is set as an initial value of an STC 
of the first decoder and an STC of the second decoder. 

30 In step 241 e, processing of the first decoder is started. 
In step 241 f, it is checked whether or not a first VPTS 
has reached the initial value. If yes, decoding is started 
in step 241g. In step 241 h, a processing delay time 
period of the first decoder is calculated, and the VPTS 

35 of the decoder output is adjusted so that the APTS and 
the VPTS are synchronized with each other. Since the 
second decoder is processed in the same manner, the 
picture from the first decoder and the picture from the 
second decoder are synchronized with each other. 

40 Thus, the decoder outputs, i.e., the first MPEG signal 
and the second MPEG signal are synchronized within 
one line. Then, the synchronization on a dot-by-dot 
basis is obtained by a video signal synchronization sec- 
tion 36a of the synthesis section 36. An original progres- 

45 sive picture is obtained even by a sum calculation. As 
shown in Figure 5, in the case where an APTS 84 is 
read by the audio decoder 16c and an identical APTS is 
set in registers 39a and 39b of the STCs of the two 
MPEG decoders 16a and 16b, an audio stream and the 

so two video stream are automatically synchronized with 
one another. 

[0113] In the present invention, when the buffer cir- 
cuits 23a and 23b underflow, either one of the pictures 
is disconnected, as a result of which a disturbed pro- 
55 gressive picture is output. In order to avoid this, the 
buffer amounts of the two buffer circuits are controlled 
by a buffer amount control section 23c as shown in Fig- 
ure 2. This operation is illustrated in the flowchart shown 
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in Figure 27. First, in step 240a, a maximum interleave 
value among the NAVI information of each disk is read, 
and a maximum value of 1 ILB in one main interleave 
block is set. The maximum value is usually 512 sectors, 
i.e., about 1 MB. When the maximum value is set below 

I MB by a specific format, that value is set as the maxi- 
mum value. Next, when an instruction to simultaneously 
reproduce the main and sub interleave blocks is issued 
in step 240b, if the buffer amount of the first buffer circuit 
23a is 1 ILB or less in step 240c, an instruction to repro- 
duce the data from the main interleave block and trans- 
fer the data to the first buffer circuit 23a is issued. Then, 
the processing goes back to steps 240b and 240c. The 
transfer is stopped in step 240d when the buffer amount 
of the first buffer circuit exceeds 1 1LB. Since the data in 
the buffer circuit 23a becomes 1 1LB or more in this man- 
ner, underflow is prevented. 

[01 1 4] In step 240f, a maximum value of a sub inter- 
leave block of 1 ILB-Sub is set in the buffer circuit 23b. 
Simultaneous reproduction is performed in step 240g. 
When the data in the second buffer circuit 23b is 1/2 
ILB-Sub or less in step 240h, data is read into the buffer 
circuit in step 240j. When the data is more than 1/2 ILB- 
Sub, the reading is stopped in step 240i. 
[0115] As shown in part (4) of Figure 24, the data 
amount of 1/2 ILB is sufficient in the second buffer cir- 
cuit. Accordingly, the buffer amount can be reduced to 
half. The buffer control in Figure 27 eliminates the 
underflow of the buffer circuits, thus reducing distur- 
bance in the synthesized picture during reproduction. 

(Required capacity of the track buffer: Figures 23 and 
31) 

[0116] First, a method for synchronizing two video 
streams according to the present invention will be 
described. First, as shown in Figure 39, a system repro- 
duced from the optical disk is once accumulated in a 
track buffer 23 and then sent to a first video decoder 
69d and a second video decoder 69c. In the track of the 
optical disk, a first stream A and a second stream B of 
the progressive signal are alternately recorded on an 
interleave block-by-interleave block basis. 
[0117] First, the stream A is reproduced at 2X rota- 
tion, and data accumulation in a first track buffer 23a in 
the track buffer 23 is started. As shown in part (1 ) of Fig- 
ure 24, when t=t1 to t2, data for 1 interleave block (ILB) 

II of the first video signal for 1 interleave time T1 is 
accumulated. A first track buffer data amount is 
increased, and becomes equal to 1 ILB at t=t2. Thus, 
data accumulation for 1 ILB of the first video signal is 
completed. At t=t2, after accumulation of data for 1 ILB 
of the first video signal corresponding to 1 GOP or more 
is completed, the second video signal (stream B) is 
reproduced from the optical disk starting from the inter- 
leave block 12. As shown in the solid line in part (4) of 
Figure 24, data accumulation of the second video signal 
in a second track buffer 23b is started at t=t2 and con- 



tinued until t=t6. From t=t2 through t8, as shown in parts 
(7) and (10) of Figure 24, the video presentation time 
stamps (VPTS) of the first video signal and the second 
video signal are synchronized and respectively sent to 

5 the first video decoder 69c and the second video 
decoder 69d from the track buffer 23a and the track 
buffer 23b. As shown in parts(8) and (1 1) of Figure 24, 
the input signals are output as two pieces of video data 
after being extended by the first and second video 

10 decoders 69c and 69d. The output of these pieces of 
data starts at t=t3, which is delayed by a video delay 
time period twd, which is required for MPEG extension 
of the data. From t=t4 through t1 0, the streams A and B 
are synthesized into a progressive signal by a progres- 

75 sive conversion section 170. Thus, a progressive signal 
for one interleave block is output. 

[0118] As described above, from t=t2 through t8, 
data for one interleave block is input to the decoders. 
Accordingly, the data in the first track buffer 23a and the 

20 data in the second track buffer 23b are consumed and 
reduced at substantially the same rate. Therefore, as 
shown in part (2) of Figure 24, the data amount in the 
first track buffer is reduced from t=t2 through t7. At t=t7, 
the data amount is 1/2 of 1 ILB. Since data reproduction 

25 for the interleave block 15 starts at t=t7, the data amount 
increases until t=t8, when the data amount reaches 1 
ILB. Since data input to the first decoder 69c starts at 
t=t8 as at t=t2, the data amount reduces until t=t11. 
Finally, the buffer memory amount becomes 1/2 ILB. 

30 [0119] With reference to part (4) of Figure 24, a 
change in the memory amount in the second track 
buffer 23a for stream B will be described. At t=t2, input 
of data B1 for the interleave block 12 of stream B in the 
second track buffer 23b starts. At the same time, trans- 

35 fer of data B1 to the second video decoder 69d starts. 
Accordingly, the buffer amount at t=t6 is 1/2 ILB. When 
2-angle recording of a progressive signal according to 
the present invention is performed, it is necessary to 
perform a track jump to the interleave block 15 over the 

40 interleave blocks 13 and 14 from time t=6 to t=7 since 
there are four streams, i.e., four interleave blocks. Dur- 
ing the jump period 197 (tj), data input from the optical 
disk is interrupted. Thus, the buffer amount of the 
stream B is reduced until t=t8, when the buffer amount 

45 is close to zero. 

[0120] Since input of data B2 of the interleave block 
16 starts at t=t8, the buffer amount starts increasing 
again. At t=t1 1 , the memory amount of the second track 
buffer is 1/2 ILB. At t=t1 1, a track jump to the interleave 

so block 19 of A3 over the interleave blocks 17 and 18 is per- 
formed. 

[0121] The above-described operation is repeated. 
[0122] Now, the minimum necessary memory 
capacity for a track buffer 23 (total capacity of the first 
55 and second track buffers 23a and 23b) according to the 
system of the present invention will be described. A 
track buffer capacity 198 indicated by the dotted line in 
part (4) of Figure 24 shows the total data amount in the 
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first and second track buffers 23a and 23b. A continu- 
ous reproduction is realized by setting the total capacity 
of a minimum 1 ILB in the track buffer. 
[0123] According to the present invention, the total 
capacity of the track buffers 23a and 23b is set to be 1 
interleave block or more for reproduction of a progres- 
sive signal. Thus, overflow and underflow of the track 
buffer are prevented. 

(Method for control the system clock) 

[0124] A method for switching the system clock 
STC between two streams will be described with refer- 
ence to Figure 28. A progressive signal includes two 
streams A and B. Here, the streams of two interlace sig- 
nals forming a 1 ILB progressive signal are referred to 
as A1 and B1 . As shown in part (1 ) of Figure 28, data A1 
for stream A is reproduced during the 1/2 ILB time 
period and all the data is recorded in the buffer. Then, 
as shown in part (2) of Figure 28, data for stream B is 
reproduced as B1 and stored in the buffer after A1 is 
reproduced. Since the data reproduced from the optical 
disk is restricted with stream B (part (2) of Figure 28) as 
described above, the track buffer does not overflow. 
Stream A (part (3) of Figure 28) or stream clock (SCR) 
from the track buffer for stream B is reset substantially in 
synchronization with the start point J of the reproduction 
of stream B (part (2) of Figure 28). Since stream B is 
output at the rate of 2X, the stream clock is counted at 
the rate of 1X as shown in part (3) of Figure 28, i.e., at 
half the rate of stream B due to the buffer. At point G, the 
stream clock is reset. Time VPTS2 at which the video 
signal for stream B is output from the video decoder 
needs to be synchronized in consideration of the delay 
time period Tvd due to, for example, MPEG decoding 
time period. In this case, at point I (t=Ti), when the 
VPTS stops rising, AV synchronization control is 
restarted. By checking VPTS2 of stream B and synchro- 
nizing VPTS1 of stream A to VPTS2, synchronization is 
realized by one-system simple control. VPTS1 can be 
used additionally. 

[0125] Audio data of synchronizing stream B is 
reproduced and the system clock is switched at point H 
using APTS of stream B as shown in part (4) of Figure 
28. Regarding a sub picture signal of stream B, the STC 
can be switched in a similar manner. 
[0126] By using data of stream B with priority. AV 
synchronization is realized with simple control. 
[0127] Since all the data in streams A1 and A2 is 
stored in the buffer memory, the buffer memory does not 
overflow. Stream B1 may possibly overflow. However, 
according to the present invention, the synchronization 
control is performed using stream B and thus the sys- 
tem clock is switched to control the signal flow so that 
VPTS2 does not exceed the VPTS2 threshold level as 
shown in part (6) of Figure 28. Therefore, the buffer 
does not overflow. 

[0128] According to the present invention, the audio 



signal of stream B is used for audio reproduction. 
Therefore, the buffer amount of audio decoder is 
reduced to 1/2. Furthermore, by switching the system 
clock at point H(t=Th) as shown in part (4) of Figure 28, 

5 the audio signal is reproduced smoothly without 
exceeding the APTS threshold level. The sub picture 
information is also reproduced with smooth synchroni- 
zation. Accordingly, picture, audio and sub picture (sub- 
titles or the like) signals are synchronized, and picture 

10 and audio are reproduced seamlessly with no interrup- 
tion. The audio signal and the sub picture signals of 
stream A can be omitted. 

(AV synchronization: Figures 29, 30, 31 and 33) 

15 

[0129] AV synchronization, which is especially 
important for connection and the like when a jump is 
performed to reproduce two or three streams simultane- 
ously, will be described. This is important in the present 
20 invention, according to which the streams of the 720P 
signal and the 480i signal, which are significantly differ- 
ent from each other in the data amount, are synchro- 
nized. 

[01 30] Figure 29 is a flowchart showing the detailed 

25 process of reproduction of a program chain group per- 
formed by the system control section 21. As shown in 
Figure 29, in steps 235a, 235b and 235c, the system 
control section 21 reads corresponding program chain 
information from the volume information file or a pro- 

30 gram chain information table of the video file. When the 
program chain is not completed in step 235d, the 
processing advances to step 235e. 
[0131] In step 235e, it is determined whether or not 
the current cell and the immediately previous cell should 

35 be connected seamlessly referring to seamless connec- 
tion instruction information for the cell to be transferred 
next in the program chain information. If seamless con- 
nection is necessary, the processing goes to step 235f 
for seamless connection processing. If not, ordinary 

40 connection is performed. 

[0132] In step 235f, the mechanism control section 
and the signal processing section, for example, are con- 
trolled to read DSI packets, so that VOB reproduction 
end time (VOB_E_PTM) in the DSI packet of the cell 

45 which has been transferred and VOB reproduction start 
time (VOB_S_PTM) in the DSI packet of the cell to be 
transferred next are read. 

[0133] In step 235h, "VOB reproduction end time 
(VOB_E_PTM) - VOB reproduction start time 

50 (VOB_S_PTM)" is found by calculation. The resultant 
value is sent to an STC offset synthesis section 164 in 
the AV synchronization control section 158 in Figure 30 
as an STC offset value between the current cell and the 
immediately previous cell which has been transferred. 

55 [0134] Simultaneously, in step 235i, VOB reproduc- 
tion end time (VOB_E_PTM) is transferred to an STC 
switch timing control section 166 as switching time T4 
for an STC switch 162e. 
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[0135] The system control section 21 then instructs 
the mechanism control section to continue reading data 
up to the terminal position of the current cell. Thus, the 
data for the current cell is transferred to the track buffer 
23 in step 235j. Upon completion of the transfer, the 
program chain information is read in step 235c. 
[0136] If it is determined the seamless connection 
is not necessary in step 235e, the data is transferred to 
the track buffer 23 up to the end of the system stream, 
and then program chain information is read in step 
235c. 

[0137] Hereinafter, two examples of a method for 
AV synchronization control for seamless connection to 
perform seamless reproduction will be described. In 
other words, the AV synchronization control section 158 
shown in Figures 2 and 31 will be described in detail. 
[0138] Referring to Figure 31, a system decoder 
161, an audio decoder 160, video decoders 69c and 
69d, and a sub picture decoder 159 are all synchro- 
nized to a system time clock given by the AV synchroni- 
zation control section in Figure 30 to process the data in 
the system stream. 

[0139] Regarding a first method, the AV synchroni- 
zation control section 158 will be described with refer- 
ence to Figure 30. 

[0140] In Figure 30, the AV synchronization control 
section includes STC switches 162a, 162b, 162c and 
162d, an STC 163, an STC offset synthesis section 
164, an STC setting section 165 and an STC switch tim- 
ing control section 166. 

[0141] The STC switches 162a, 162b, 162c, 162d 
and 162e switch between an output value of the STC 
163 and an output value of the STC offset synthesis 
section 164 as a reference clock to be provided to the 
system decoder 161, the audio decoder 160, the main 
video decoder 69c, the sub video decoder 69d and the 
sub picture decoder 159, respectively. 
[0142] The STC 163 is a reference clock of the 
entire MPEG decoder shown in Figure 31 in ordinary 
reproduction. 

[01 43] The STC offset synthesis section 1 64 contin- 
ues outputting a value obtained by subtracting the STC 
offset value provided by the system control section from 
the value of the STC 163. 

[0144] The STC setting section 165 sets an STC 
initial value given by the system control section or an 
STC offset synthesis value given by the STC offset syn- 
thesis section 164 in the STC 163 at the timing given by 
the STC switch timing control section 166. 
[0145] The STC switch timing control section 166 
controls the STC switches 162a through 162e and the 
STC setting section 165 based on STC switch timing 
information given by the system control section, the 
STC 163, and the STC offset synthesis value given by 
the STC offset synthesis section 164. 
[0146] The STC offset value is an offset value used 
for changing the STC value when system stream #1 and 
system stream #2 having different STC initial values are 



continuously reproduced. 

[0147] The STC offset value is specifically obtained 
by subtracting the "VOB reproduction start time 
(VOB_S_PTM)" described in the DSI of system stream 

5 #2 to be reproduced next from the "VOB reproduction 
end time (VOB_E_PTM)" described in the DSI packet of 
system stream #1 reproduced first. The information 
regarding the display of such a value is pre-calculated 
by reading data from the optical disk in Figure 5 by the 

10 system control section 167 when the data is input to the 
track buffer 23. 

[0148] The calculated offset value is supplied to the 
STC offset synthesis section 164 before the last pack of 
system stream #1 is input to the system decoder 161. 

15 [0149] Except for seamless connection control, the 
data decoding processing section 165 in Figure 5 oper- 
ates as an MPEG decoder. The STC offset value given 
by the system control section 21 is 0 or an arbitrary 
value. The STC switches 162a through 162e are always 

20 selected to be connected to the STC 163. 

[0150] With reference to the flowchart in Figure 33, 
switching of the STC switches 162a through 162e in the 
connection part of the system control section and an 
operation of the STC 163 when two system streams 

25 having non-continuous STC values, such as system 
streams #1 and #2, are continuously input to the system 
decoder 161, will be described. 

[0151] The SCR, APTS, VPTS and VDTS of the 
system streams #1 and #2 to be input will not be 

30 described. 

[0152] It is assumed that in the STC 163, an initial 
STC value corresponding to system stream #1 which is 
being reproduced is set by the STC setting section 165, 
and the value is sequentially counted up in accordance 

35 with the reproduction. The system control section 21 
(Figure 31) calculates the STC offset value by the 
above-described method and sets this value in the STC 
offset synthesis section 164 before the last pack of sys- 
tem stream #1 is input to the decoder buffer. The STC 

40 offset synthesis section 164 continues outputting a 
value obtained by subtracting the STC offset value from 
the value of the STC 163 (step 168a). 
[0153] The STC switch timing control section 166 
obtains time T1 , at which the last pack of system stream 

45 #1 reproduced first is input to the decoder buffer, and 
switches the STC switch 162a to the output side of the 
STC offset synthesis section 164 at time T1 (step 
168b). 

[0154] Thereafter, the STC value referred to by the 
50 system decoder 161 is provided with an output from the 
STC offset synthesis section 164. The transfer timing of 
system stream #2 to the system decoder 161 is deter- 
mined by the SCR described in the pack header of sys- 
tem stream #2. 

55 [0155] Next, the STC switch timing control section 
166 obtains time T2, at which the reproduction of the 
last audio frame of system stream #1 reproduced first is 
terminated, and switches the STC switch 162b to the 
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output side of the STC offset synthesis section 164 at 
time T2 (step 168c). A method for obtaining time T2 will 
be described later. 

[0156] Thereafter, the STC value referred to by the 
audio decoder 160 is provided with an output from the 
STC offset synthesis section 164. The audio output tim- 
ing of system stream #2 is determined by the APTS 
described in the audio packet of system stream #2. 
[0157] Next, the STC switch timing control section 
166 obtains time T3 and T3', at which the decoding of 
the last video frame of the main signal and the sub sig- 
nal of system stream #1 reproduced first is terminated, 
and switches the STC switches 162c and 162d to the 
output side of the STC offset synthesis section 164 at 
time T3 and T3' (step 168d). A method for obtaining 
time T3 will be described later. Thereafter, the STC 
value referred to by the video decoders 69c and 69d is 
provided with an output from the STC offset synthesis 
section 164. The video decoding timing of system 
stream #2 is determined by the VPTS described in the 
video packet of system stream #2. 
[0158] Next, the STC switch timing control section 
1 66 obtains time T4, at which the reproduction output of 
the last video frame of system stream #1 reproduced 
first is terminated, and switches the STC switch 162e to 
the output side of the STC offset synthesis section 164 
at time T4 (step 168e). A method for obtaining time T4 
will be described later. 

[0159] Thereafter, the STC value referred to by the 
video output switch 169 and the sub picture decoder 
159 is provided with an output from the STC offset syn- 
thesis section 164. The video output timing and sub pic- 
ture output timing of system stream #2 are determined 
by the VPTS and SPTS described in the video packet 
and the sub picture packet of system stream #2. 
[0160] When switching of the STC switches 162a 
through 162e is completed, the STC setting section 165 
sets the value given by the STC offset synthesis section 
164 in the STC 162 (step 168f) (referred to as "reload- 
ing of the STC 163) and switches all the switches 162a 
through 162e to be connected to the STC 163 (step 
168g). 

[0161] Thereafter, the STC value referred to by the 
audio decoder 160, the video decoders 69c and 69d, 
the video output switch 169 and the sub picture decoder 
159 is provided with an output from the STC 163, and 
the operation returns to the ordinary operation. 
[0162] Now, two means for obtaining time T1 
through T4 for switching the STC will be described. 
[0163] According to specific means, information 
representing time T1 through T4, which can be easily 
calculated when the streams are created, is recorded 
on the disk. The system control section 21 reads the 
information and sends the information to the STC switch 
timing control section 166. 

[0164] Especially for T4, "VOB reproduction end 
time (VOB_E_PTM) M described in the DSI used for 
obtaining the STC offset is used as it is. 



[01 65] On the disk, the value obtained based on the 
STC value used in system stream #1 reproduced first is 
described, and the STC switch timing control section 
166 switches the STC switches 162a through 162e at 
5 the moment the value of the STC 163 becomes time T1 
through T4. 

(Example 2) 

w [0166] In the first example, an example of applica- 
tion of a system for reproducing a plurality of streams in 
synchronization according to the present invention is 
described in detail. In a second example, this system is 
applied to a reproduction control system for reproducing 

15 two streams seamlessly. In the case of recording an 
MPEG signal, editing is conventionally performed on a 
GOP-by-GOP basis in general, and it is difficult by con- 
ventional methods to perform editing on a frame-by- 
frame basis. By using an MSS system according to the 

20 present invention, substantial frame-based editing is 
realized. 

[0167] It is important to synchronize the timing of 
the video signal and the audio signal at the point of con- 
nection. Specific synchronization systems will be 

25 described in third through ninth examples. 

[0168] The reproduction control system according 
to the present invention can be applied as follows to 
connect two streams while switching the two streams on 
a frame-by-frame basis seamlessly. As shown in Figure 

30 10, editing data 761 including synthesis information of a 
28P zoom instruction signal and the like is processed by 
an editing data processing section 762 and sent to a 
switching synthesis section 763. The data is then 
switched/synthesized and output from a switching syn- 

35 thesis signal output section 764. Thus, two MPEG video 
signals can be switched and connected seamlessly at 
an arbitrary point other than the borders between 
GOPs. 

[0169] A system for synthesizing two pictures at an 
40 arbitrary point based on an instruction signal will be 
described later with reference to Figure 22. 
[0170] In a simple switching mode, two pictures are 
simply switched over frame by frame at an editing point 
as follows. Stream "a" and stream "b" are switched over 
45 at an editing point tc, and the resultant stream is output 
seamlessly. In a synthesis switching mode (such as a 
wipe), stream "a" and stream "b" are switched after 
being synthesized from a start point ts to a termination 
point te. As shown in Figures 6 and 58, in mode 1, 
so switching is performed from left to right; in mode 2, from 
center to periphery; in mode 3, top to bottom; and in 
mode 4, in a mosaic manner. Figure 6 is a simplified 
block diagram; and Figure 58 is a detailed block dia- 
gram. 

55 [0171] In Figure 6, reproduction means 778, a divi- 
sion section 734, a VTS synchronization section 780, 
and MPEG decoders 728 and 730 have exactly the 
same structure as that of those in the 480P reproduc- 
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tion apparatus shown in Figure 3, and thus the 480P 
reproduction apparatus can be used. With reference to 
Figure 6, when an identification information processing 
section 766 detects reproduction control information 
766a, two video streams are sent to the switching syn- 
thesis section 763, and the first stream is switched to 
the second stream seamlessly at the connection point ts 
as described above. 

[0172] With reference to Figure 58, when a 
720P/480P hierarchical recording identifier 725 having 
a high resolution signal described in the first example is 
detected, the synthesis section 732a performs calcula- 
tion and outputs a high resolution signal such as a 480P 
or 720P signal. 

[0173] When a 3D recording identifier 766c is 
detected a 3D signal processing section 770 generates 
a 3D video signal having a right-eye video signal and a 
left-eye video signal alternately interleaved, and outputs 
the signal. 

[0174] In this manner, the MSS system shown in 
Figure 58, which uses two MPEG decoders or an 
MPEG decoder for decoding two streams simultane- 
ously, can be used to provide three functions, i.e., repro- 
duction control with frame-based editing, reproduction 
of a high resolution signal, and reproduction of a 3D 
video signal. 

[01 75] Figure 1 1 shows a specific example of repro- 
duction control information 765. The reproduction con- 
trol information 765 includes a switching point number S 
766, a synthesis mode identifier 767, a first stream 
switching start address ts 768, a first stream switching 
termination address te2 769, a second stream GOP 
start address tsG 790, a second stream switching start 
address ts2 771, and a second stream switching termi- 
nation address te2, 772 in the order of the address 
used. 

[0176] Specifically, when the switching point 
number S=1 , as shown in part (9) of Figure 12, a picture 
synthesis identifier 767 is not present or is 0. Accord- 
ingly, the first stream is simply switched into the second 
stream at the switching address of ts1 -1 . When S=2, as 
shown in part (10) of Figure 12, switching is started at 
ts1 , and the two pictures of the first stream and the sec- 
ond stream are synthesized into one picture until t=te1 . 
At t=te1 , the first stream is completely switched into the 
second stream. 

[0177] With reference to the flowchart of Figure 13, 
a process for reproduction performed based on the 
reproduction control information will be described. 
[0178] With reference to part (1) of Figure 12, a 
GOP 781a of the first stream and a GOP 781b of the 
second stream may be at discrete positions on an opti- 
cal disk without the streams being moved or rewritten. In 
this case, the time period for rewriting can be saved. 
With reference to part (3) of Figure 12, since data can 
be recorded on a DVD-RAM disk and the like after 
edited, streams are recorded on a GOP-by-GOP basis; 
i.e., a GOP 781e and a GOP 781f both including a 



frame at the editing point are recorded adjacent to each 
other. In this case, the editing point is allowed to be 
changed in one GOP later. When two streams are syn- 
thesized in a wipe manner, the picture of the first stream 

5 after the editing point is required. Thus, the structure 
shown in part (3) of Figure 12 is required. 
[0179] When S=0, i.e., when synthesis of two pic- 
tures is not performed, the data after the switching point 
ts1 of a GOP 781c is not required and thus is deleted as 

10 shown in part (2) of Figure 12 to exclude the redundant 
portion. Thus, the recording efficiency is raised. How- 
ever, a GOP 781 d, which has the IN point, includes an i 
(intra) frame (i.e., basic frame) at the beginning, which 
cannot be deleted. Thus, a redundant portion 783 is 

15 generated. 

[0180] As shown in step 792a in Figure 59, 1GOP 
includes about 15 actual frames. When the B frames 
present before the IN point are deleted as in step 792b, 
the number of frames is reduced from 12 in step 792a 

20 (redundant portion 783f) to 3 in step 792c (redundant 
portion 783g). The redundant portion 783g is about 1/4 
of the redundant portion 783f, and thus the recording 
efficiency is raised. 

[0181] When this portion is reproduced, a B frame 

25 deletion identifier is detected in step 792f. The number 
of frames are calculated with a premise that the B 
frames are not recorded. Since an MPEG signal is 
decoded with only I, P frames, the frames are decoded 
one after another in step 792g. The frame having the IN 

30 point (i.e., t=ts2 ) is obtained by decoding and output. In 
this case, the number of frames to be processed is only 
three. Accordingly, the intended IN point can be repro- 
duced in a 1/4 time period. In this example, the redun- 
dant portion is 1/8 second long. As can be appreciated 

35 from Figure 59, ts2 is in the 14th frame in the worst 
case. In this case, the redundant portion is five frames 
long of I, P, P, P, B, i.e., 1/3 x 1/2 = 1/6 second long. That 
is, the longest possible redundant portion is about 0.18 
second. At least such a time period is necessary to 

40 reproduce the IN point. Even if about five cut portions 
are included in one second, each cut portion is present 
every 0.2 second. By deleting the B frames by this sys- 
tem, the frame-based reproduction is realized even if 
there are five cut portions in one second. This means 

45 that this system is usable in the standard editing with no 
problem. 

[0182] A method for generating reproduction con- 
trol information will be described. Where the final GOP 
before the OUT point is defined as a first GOP and the 

50 first GOP after the IN point is defined as a second GOP, 
reproduction control can be performed simply by 
recording the time of the beginning of the second GOP 
as the switching point ts2 and the time of the switching 
point. Alternatively, the number of frames from the 

55 beginning of the second GOP to the switching point can 
be recorded. 

[0183] When such reproduction control information 
is reproduced by a reproduction apparatus, the frame at 
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the switching point is decoded without processing the B 
frames (pictures) among I, B, B, B, R B, B, B, P, B, B, B 
as shown in step 792f in Figure 59. In other words, only 
I, R P are decoded. In this manner, the picture at the IN 
point is obtained in a 1/4 time period as described 
above. Even if there are five frame switching points in 
one second, the redundant time period is 0.18 second, 
which is shorter than the pitch of the cut portion (0.2 
second). Thus, the data can be reproduced seamlessly 
at all the switching points. 

[0184] In order to realize synchronization, repro- 
duction control intonation is obtained by calculating the 
number of frames (pictures) existing between the sec- 
ond GOP and the switching point based on the time of 
the beginning of the second GOP and the time of the 
switching point. When the B frames which are not nec- 
essary are deleted, the correction is performed in con- 
sideration of the deleted frames referring to an 
unnecessary frame deletion identifier. Then, it can be 
found how many frames should exist between the start 
of reproduction of the second GOP, the IN point of the 
second GOR and the OUT point of the first GOP, in 
order to synchronize the OUT point of the first GOP and 
the IN point of the second GOP. 

[0185] When the number of frames existing 
between the beginning of the second GOP and the 
switching point is recorded as the reproduction control 
information, correction in consideration of the unneces- 
sary frames such as the B frames can be performed in 
order to find the time to start decoding the second GOP 
to realize the synchronization. 

[0186] Another recordable reproduction control 
information can be decoding start timing information 
which indicates a specified point in the first GOP at 
which decoding of the second GOP should be started in 
order to match the switching point of the first GOP and 
the switching point of the second GOP. 
[0187] In such a case, the switching points can be 
synchronized using only the reproduction control infor- 
mation, without performing any special calculation by 
the reproduction apparatus. 

[0188] The redundant portion 783 includes one I 
frame, a plurality of P frames and a plurality of B frames. 
By decoding these frames to create a frame immedi- 
ately before the frame corresponding the final editing 
point (i.e., intraframe), the recording efficiency can fur- 
ther be raised. In the case of a DVD-RAM disk, the over- 
all editing structure is found as toe by recording the 
overall reproduction control information 765 and limited 
reproduction control information 765a on only the 
switching points, at two points, i.e., at the beginning of 
the recording data and immediately before the editing 
point as shown in parts (1), (2) and (3) of Figure 12. 
Before the editing point, limited reproduction control 
information 765a on each separate editing point, for 
example, S=1 can be recorded among the overall repro- 
duction control information. In this case, reproduction 
control for special reproduction is advantageously stabi- 



lized. 

[0189] A process for reproduction control will be 
described. First, in step 774a, reproduction control infor- 
mation is read. In step 774b, the reproduction switching 

5 point number S is set to 0. In step 774c, S is incre- 
mented by one. The decoding start position of the sec- 
ond stream needs to be specified. In step 774d, it is 
checked whether or not t=ta , where t is the system 
clock or the VPTS of the first stream and ta is the decod- 

w ing start position information. When t=ta , i.e., when the 
VPTS of the second stream reaches ta as shown in part 
(5) of Figure 12, the processing advances to step 774e, 
where MPEG-decoding of the GOP of the second 
stream is started as shown in part (6) of Figure 12. In 

15 step 774f, it is checked whether or not t=ts1 . As shown 
in parts (5) and (6) of Figure 12, the switching point ts2 
comes after the time corresponding to the value of 
(ts1 -ta)=(ts2-tsG) passes. As shown by the expression 
and parts (5) and (6) of Figure 12, the data at ts1 of the 

20 first stream and data at ts2 of the second stream are 
MPEG-decoded at the same time and output from the 
decoder. Parts (8) and (9) of Figure 12 show the state 
where the two streams are synchronized. Specific 
methods for synchronization will be described in detail 

25 in the third through ninth examples. It is appreciated 
from the reproduction control information that ts1 and 
ts2 are frame editing points of the two pictures. In step 
774g, it is determined whether or not there is a picture 
synthesis identifier 767. If the picture synthesis identifier 

30 767 does not exist, the processing advances to step 
774h, where the first stream is switched into the second 
stream by the switching synthesis section 763 (Figure 6) 
at the position of t=ts1 as shown in part (10) of Figure 
12. Since t=ts1 is the OUT point in actuality, the frame 

35 of the first stream at t=ts1 is not output. Since t=ts2 is 
the IN point, the frame of the second stream at this point 
is output. Since the frame information on the first stream 
is not necessary at t=ts2 , recording of such information 
can be omitted when S=0, i.e., when no picture synthe- 

40 sis is performed. In this case, the recording efficiency is 
raised by one P frame. Thus, reproduction control of the 
switching point for S=1 (simple switching mode) is com- 
pleted. The processing returns to step 774c and deals 
with the switching point of S=2. 

45 [0190] If n=0 in step 774P, it is converted into n=1. 
In other words, when the second stream is decoded by 
the second MPEG decoder 730, the second stream, 
i.e., the MPEG signal to be switched next is decoded by 
the first MPEG decoder 728. A different MPEG decoder 

50 is used from the case of S=1 . 

[0191] The second stream is decoded as shown in 
Figure 56. Input second streams 781a, 781b, 781g, 
781 h and 781 i are sent by the division section 734 to 
the first MPEG decoder 728 and the second MPEG 

55 decoder 730 alternately and decoded into video signals 
788a and 788b. The video signals 788a and 788b are 
synthesized into one stream by the switching section 
763. As is clear from the figure, the output from the first 
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MPEG decoder 728 is stopped after the decoded video 
signal 788a is output. That is, the picture is frozen. This 
occurs since non-continuous data cannot be normally 
decoded. The MPEG decoding is resumed by perform- 
ing resetting 790 of registers and the like. The second 
MPEG decoder is processed similarly. Conventionally, 
since only one MPEG decoder is used to perform seam- 
less reproduction, various complicated pre-processing 
needs to be performed at the time of recording. Accord- 
ing to the present invention, such various complicated 
processing is not necessary at the time of recording for 
the following reason. Since even if one of the MPEG 
decoders is stopped, the other MPEG decoder can be 
used, even MPEG data which could not otherwise be 
connected seamlessly can be reproduced seamlessly. 
While the second MPEG decoder operates, the first 
MPEG decoder is processed to resume operating. In 
this manner, seamless reproduction can be continued 
endlessly by switching the two MPEG decoders. Even 
frame-based editing of MPEG signals, which conven- 
tionally requires complicated processing, can be real- 
ized without complicated processing by using 
reproduction control according to the present invention. 
The system according to the present invention does not 
require the step of decoding and re-encoding MPEG 
data and thus provides a significant effect that the pic- 
ture quality is not deteriorated. 

[0192] The process for seamless reproduction can 
be summarized as follows. A plurality of streams are 
divided and alternately decoded by two MPEG decod- 
ers. An output from one of the MPEG decoders is 
switched into an output from the other MPEG decoder 
at a switching point. While the other MPEG decoder is 
outputting data, the first MPEG decoder is reset to 
decode the next stream. At the next switching point, the 
output from the second MPEG decoder is switched into 
the output from the first MPEG decoder. 
[0193] Thus, continuous frame-based reproduction 
is realized. 

(Synthesis switching mode) 

[0194] Returning to step 774g of Figure 13, a proc- 
ess for switching two streams while synthesizing the two 
streams into one picture in for example, a wipe manner, 
by the synthesis signal output section 764 (shown in 
Figure 6) will be described. In this case, the picture syn- 
thesis identifier 767 is not 0. In step 774i, the first 
stream is switched into the second stream as shown in 
part (1 1 ) of Figure 12 while the two streams are synthe- 
sized from t=ts1 . The switching is continued until the 
time becomes t=te1 or t=te2 in step 774j. The switch- 
ing is completed in step 774k. Since the decoding of the 
first stream is stopped at t=ts1 , wasteful decoding of 
data is prevented. The switching is performed from left 
to right as in the screen 782a in mode 1, from center to 
periphery as in the screen 782b in mode 2, top to bot- 
tom as in the screen 782c in mode 3, and in a mosaic 



manner as in the screen 782d in mode 4. 
[0195] Figure 12 shows an example in which the 
original time stamp is not changed. When the recording 
is performed using a DVD-RAM disk or the like where 

5 the time of the switching point of the first stream 
matches the time of the switching point of the second 
stream, the time stamp can be changed and thus the 
structure is further simplified. The time stamp can be 
changed at the time of recording by setting ts1=ts2. In 

w this case, the first stream can be recorded up to tel. 
Time stamp tsG of the second stream is changed to 
ts1 -(ts2-tsG) . A time stamp having a smaller value than 
ts1 is assigned. Time ta is identical with tsG. Accord- 
ingly, when reproduction is performed as shown in part 

15 (6) of Figure 12, tsG, i.e., the GOP start address 770a 
of the second stream is pre-read based on the repro- 
duction control information 765b shown in Figure 14, 
and decoding is stated at tsG. 

[0196] When a system of changing the time stamp 
20 is used, the time stamp of a GOP 761 e and the time 
stamp of a GOP 781 f in part (3) of Figure 12 are 
changed in the order and also overlapped with each 
other. When such a stream is reproduced, the reproduc- 
tion apparatus malfunctions. According to the present 
25 invention as shown in Figure 14, the GOP start address 
770a of the second stream is recorded in the reproduc- 
tion control information 765b. Thus, the positions of the 
time stamps which are changed in the order and also 
overlapped are found in advance. Therefore, malfunc- 
30 tion can be advantageously prevented even while fast- 
forwarding. Use of the GOP start address 770a further 
secures the switching into the second stream at the 
editing point. 

[0197] When the reproduction is performed as 
35 shown in part (6) of Figure 12, tsG, i.e., the GOP lead- 
ing address 770a of the second stream is pre-read 
based on the reproduction control information shown in 
Figure 14, and decoding is started at tsG. 
[0198] Alternatively, when information on the editing 
40 point ts1 is available, the leading address tsG of the 
GOP including the editing point is pre-read and the data 
on the GOP 781 f of the second stream is MPEG- 
decoded at t=tsG . This provides an advantage that the 
editing point of the first stream and the editing point of 
45 the second stream automatically match at ts1 . 

[0199] By recording data with the time stamps 
being changed as above, the structure and operation 
can be significantly simplified. 

[0200] The above-mentioned second recording 
50 method will be described with reference to Figures 15 
and 16. 

[0201] With reference to Figures 6 and 16, an edit- 
ing/reproduction control information generation pro- 
gram will be described. As shown in Figure 6, editing 
55 information 780 including the IN and OUT points (edit- 
ing points) is input manually or based on data and 
sequentially converted into reproduction control infor- 
mation 765 by a reproduction control information gener- 



21 



41 



EP 1 01 1 268 A1 



42 



ation section 789. The reproduction control information 
765 is once stored in a memory 779 and recorded by 
recording means 777 of the RAM disk 724 from the 
memory 779 either after all the editing operation is com- 
pleted or immediately before the disk is removed from 5 
the apparatus. 

[0202] A process performed by the reproduction 
control information generation section 789 will be 
described in detail with reference to Figure 16. 
[0203] In step 785a, the editing information 780 is 10 
input manually or based on data sequentially. S repre- 
sents the number of the editing point, and G represents 
modes 1 through 4 in which two pictures are switched 
while being synthesized by wipe or the like. tsO repre- 
sents the start point of the first stream, ts1 represents 15 
the OUT point of the first stream, te1 represents the 
OUT synthesis completion point of the first stream, ts2 
represents is the IN point of the second stream, te2 rep- 
resents the IN synthesis completion point of the second 
stream, and tl_2 is the OUT point of the second stream. 20 
[0204] In step 785b, S=0 is set. S is incremented by 
one (step 785c), and tsO and ts1 are read (step 785d). 
In step 785e, it is checked whether or not G exists. If G 
does not exist, there is no need to synthesize pictures, 
and thus in step 785f, the first stream (S) is recorded on 25 
the optical disk 724 from tsO to ts1 . The IN point ts2 of 
the second stream (S+1) is read, and the processing 
goes into a time stamp conversion routine (step 785g). 
In the time stamp conversion routine, the leading time 
stamp tsG of the leading GOP including information cor- 30 
responding to the frame corresponding to ts2, and the 
time stamp ts2 corresponding to the final frame among 
the frames corresponding to the leading GOP are 
obtained. The value of the time stamp is reduced by the 
amount corresponding to (ts2-ts1 ) from tsG to tl_2 of all 35 
the recording data of the second stream, thereby gener- 
ating a new time stamp. In step 785i, original address 
tsG to tl_2 (tf2) of the second stream is replaced with the 
new time stamp, and the information corresponding to 
the frame immediately following the frame correspond- 40 
ing to ts1 (OUT point of the first stream), is rewritten by 
the information obtained by the new time stamp. 
[0205] In step 785w, the address tsG of the leading 
GOP of the second stream is used to perform the calcu- 
lation of ta=ts1 -(ts2-tsG) so as to find the time period ta 45 
prior to the decoding of the second stream. The result- 
ant time period is added to the limited reproduction con- 
trol information 765a and the reproduction control 
information 765. In step 785j, limited reproduction con- 
trol information 765a only on the S number is recorded 50 
in the second half of the first stream (S) as shown in part 
(2) of Figure 12. If S is terminated in step 785j, the 
processing advances to step 785m; and if S is not ter- 
minated in step 785j, the processing returns to step 
785c to repeat the steps. In step 785m, the reproduc- 55 
tion control information 765 at all the editing points 
stored in the memory 779 is recorded in the reproduc- 
tion control information recording section of the optical 



disk, where management information such as TOC and 
the like are recorded, as shown in part (3) of Figure 12. 
Thus, the editing/reproduction control information gen- 
eration program is completed. 

[0206] If a synthesis identifier G exists, in step 
785n, te1 (OUT synthesis completion point of the first 
stream) is read. In step 785p, the calculation of 
tG=te1 -ts1 is performed. If tG<tGmax in step 785q, the 
processing advances to step 785r. If tG>tGmax, the 
synthesis time period of the two streams is very long 
and exceeds the capacity of the buffer of the reproduc- 
tion apparatus. In step 785v, an error message that 
"decrease te1" is issued. When te1 is changed in step 
785w, the processing returns to step 785n, and te1 is 
decreased to be within the capacity. 
[0207] Thus, the synthesis of the connection point 
is within the capacity. Accordingly, the processing 
returns to step 785r, where the first stream is recorded 
on the optical disk from tsG to te1 . In step 785s, ts2 of 
the second stream is read; and in step 785t, the time 
stamp conversion processing routine is performed in a 
similar manner to the step 785g, to convert the time 
stamp. In step 785u, the original address tsG to tf2 (tl_2) 
of the second stream is replaced with the new time 
stamp, and the information corresponding to the frame 
immediately following the frame corresponding te1 of 
the first stream is rewritten with the information obtained 
by the new stamp. Time ta is recorded in the memory. In 
step 785j, the limited reproduction control information 
765a is recorded. In step 785k, it is checked whether or 
not processing for S=1 through S=n is completed. In 
step 785m, the overall reproduction control information 
765 is recorded on the optical disk. Thus, all the opera- 
tions are completed. 

[0208] In this manner, the first stream is recorded 
up to a connection point, and a portion after the connec- 
tion point is rewritten with a time stamp having a smaller 
value of that of the connection point, so that the connec- 
tion of the first stream and the connection point of the 
second stream match each other. Thus, the reproduc- 
tion apparatus can output a video signal including the 
two streams connected to each other on a frame-by- 
frame basis. 

[0209] In this case, two MPEG decoders are 
required. The recording and reproduction apparatus 
shown in Figure 6 includes the MPEG encoder 791 for 
MPEG-encoding the input video signal. Since an MPEG 
encoder is never used at the same time as an MPEG 
decoder and also the MPEG encoder has a processing 
capability of twice or more of the MPEG decoder, one 
MPEG processing section has a function corresponding 
to one encoder or two decoders. 

[0210] Accordingly, when the present invention is 
applied to the recording and reproduction apparatus 
including an MPEG encoder, the frame-based editing is 
realized without adding any element. 
[0211] Soft-encoding/decoding performed by a 
CPU will be described. The capability of a CPU for 
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encoding one stream corresponds to the capability for 
decoding two streams. As shown in Figure 57, two 
streams can be decoded simultaneously or in a time 
division manner. Thus, the virtual frame-based editing 
according to the present invention is realized without 
raising the processing capability of the CPU. 
[0212] With reference to the flowchart of Figure 57, 
a process for performing encoding/decoding and 
recording/reproduction using a CPU will be described. 
In step 792a of encode recording, data of m=1 to final is 
input. In step 792c and 792d, the m'th video signal is 
input. The m'th video signal is encoded to create an m'th 
stream (step 792e) and recorded on an optical disk 
(step 792f). If m is not final in step 792g, the processing 
returns to step 792c; if m is final in step 792g, the 
recording is terminated (step 792h). In step 792i of 
recording reproduction control information, editing is 
performed on a frame-by-frame basis, and the repro- 
duction control information and the above-mentioned 
various identifiers are recorded on the optical disk (Fig- 
ures 6 and 58). 

[0213] Reproduction is performed as follows. The 
reproduction control program 792j is started. Cut point 
S is reproduced from 1 to final (steps 774b and 774c). 
As shown in steps 774m and 774e, two streams are 
MP EG -decoded simultaneously or in a time division 
manner at the frame-edited point. In step 774h, one 
decoded stream is switched to the other decoded 
stream at t=ts . This operation is repeated until the final 
S is processed in step 774r. 

[0214] The CPU has a capability of MPEG-encod- 
ing one stream. This means the CPU has a capability of 
MPEG-decoding two or three streams. Accordingly, one 
CPU can MPEG-encode one stream, performs frame- 
based editing, MPEG-decodes two streams, and out- 
puts the connected stream seamlessly. The present 
invention has an effect that a part of the capability of the 
CPU which is not conventionally used is effectively 
used. 

(Example 3) 

[0215] The MADM system according to the present 
invention simultaneously reproduce a plurality of 
streams. Synchronization systems are important. 
[0216] In the first example, recording and reproduc- 
tion of high resolution video signals such as 480P and 
720P signals are described. In the second example, 
basic AV synchronization systems for reproduction con- 
trol using virtual frame-based editing are described. In 
the third through ninth examples, various methods of 
synchronization will be described in more detail. 
[0217] In the third example, an operation of a repro- 
duction apparatus for reading data from an optical disk 
having three compression video signals to be repro- 
duced simultaneously, and extending and reproducing 
the three compression video signals simultaneously by 
an AV synchronization system will be described. 



[0218] Figure 37 shows a data structure of an opti- 
cal disk used in the optical disk reproduction apparatus 
in the third example. 

[0219] Video signals A, B and C are MPEG-com- 
5 pressed to obtain compression video streams A, B and 
C. 

[0220] The compression video streams A, B and C 
are each packeted in units of 2 kB into video packets. A 
packet header of each packet includes a stream ID for 

10 indicating which one of the compression video streams 
A through C is stored. When the packet stores a leading 
part of the video frame, the packet header also includes 
a VPTS (video presentation time stamp) as video repro- 
duction time information indicating the time to repro- 

15 duce the frame. In the third example, an NTSC signal is 
used as the video signal, and the video frame cycle is 
about 33 msec. 

[0221] On the optical disk, video packets created in 
the above-described manner are grouped into, for 

20 example, compression video signals A-1 , B-1 and C-1 
each including an appropriate number of packets based 
on the data stored, and multiplexed. 
[0222] Figure 35 is a block diagram of an optical 
disk reproduction apparatus in the third example. 

25 [0223] In Figure 35, the optical disk reproduction 
apparatus includes an optical disk 501 described above, 
an optical pickup 502 for reading data from the optical 
disk 501, signal processing means 503 for performing a 
series of signal processing such as binarization, 

30 demodulation, and error correction to the signal read by 
the optical pickup 502, a buffer memory 504 for tempo- 
rarily storing the data output from the signal processing 
means 503, division means 505 for dividing the data 
read from the buffer memory 504 into compression 

35 video, signals, and reference time signal generation 
means 506 for generating a reference time signal 506 
including a counter (not shown) for counting 90 kHz 
clocks. Reference numerals 510, 520 and 530 repre- 
sent buffer memories for temporarily storing the com- 

40 pression video signals divided by the division means 
505. Reference numerals 511, 521 and 531 represent 
video decoders for extending and reproducing the com- 
pression video signals. Reference numerals 512, 522 
and 532 represent monitors for displaying the video sig- 

45 nals. 

[0224] Figure 36 shows the structure of each of the 
video decoders 51 1 , 521 and 531 . 
[0225] As shown in Figure 36, the video decoder 
includes VPTS detection means 601 for detecting a 

50 VPTS stored in the packet header of the video packet, 
video extension means 602 for MPEG-extending the 
compression video stream, and video reproduction tim- 
ing control means 603 for comparing the reference time 
signal and the VPTS and skipping or repeating the 

55 video reproduction on a frame-by-frame basis when the 
comparison result exceeds the threshold value. 
[0226] The optical disk reproduction apparatus 
shown in Figure 35 operates in the following manner. 
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[0227] The optical pickup 502 is focus-controlled or 
tracking-controlled by servo means (not shown) to read 
a signal from the optical disk 501 and outputs the signal 
to the signal processing means 503. The signal 
processing means 503 subjects the signal to a series of 
processings including binarization, demodulation, error 
correction and the like. Then, the signal processing 
means 503 stores the resultant signal in the buffer 
memory 504 as digital data. 

[0228] The buffer memory 504 functions so that, 
even when the data supply from the optical disk 501 is 
temporarily stopped by, e.g., a wait state, the data sup- 
ply to the subsequent-stage sections is not stopped. 
[0229] The data read from the buffer memory 504 is 
divided into compression video signals A through C by 
the division means 505 and output. The division means 
identifies which of the compression video streams A 
through C is stored in each packet using the stream ID 
in the packet header of the packeted data, and deter- 
mines the destination based on the identification result. 
[0230] The divided compression video signals are 
respectively stored in buffer memories 510 through 530. 
[0231] The buffer memories 510 through 530 act to 
continuously supply data to the video decoders 511 
through 531. 

[0232] The video decoders 511 through 531 read 
data from the buffer memories 510 through 530 respec- 
tively, extend the compression video signals, and output 
the signals as video signals to the monitors 512 through 
532 respectively. 

[0233] With reference to Figure 36, operation of the 
video decoders 511 through 531 will be described. 
[0234] The compression video signal read from the 
buffer memory is input to the VPTS detection means 
601 and the video extension means 602. 
[0235] The video extension means 602 MPEG- 
extends the compression video stream and outputs the 
video signal. 

[0236] The VPTS detection means 601 detects the 
VPTS of the packet header and outputs the VPTS. 
[0237] The video reproduction timing control means 
603 receives the video signal output from the video 
extension means 602, a reference time signal and the 
VPTS output from the VPTS detection means 601, and 
compares the reference time signal and the VPTS. 
Then the difference between the two exceeds the 
threshold value, the video reproduction timing is control- 
led so that the difference between the VPTS and the ref- 
erence time signal is equal to or less than the threshold 
value. 

[0238] In the third example, 33 msec is used as the 
threshold value for video reproduction. The video repro- 
duction timing control means 603 performs the follow- 
ing. 

(reference time signal-VPTS)>33 msec.:1 frame is 
skipped. 

(reference time signal-VPTS)<-33 msec.:1 frame is 



repeated. 

[0239] In the third example, due to the precision 
error of the crystal oscillator used in the reference time 

5 signal generation means 506 and the video decoders 
511 through 531, the video decoders 511 and 531 are 
slower and the video decoder 521 is faster in terms of 
extension and reproduction relative to the reference 
time signal. Unless reproduction timing is adjusted, the 

w reproduced video signals are out of synchronization. 
[0240] Figure 38 is a timing diagram of video repro- 
duction in the third example. Part (a) of Figure 38 shows 
the reference time signal with respect to reproduction 
time t. Part (b) shows the VPTS#A, which is a VPTS of 

15 the compression video signal A to be extended by the 
video decoder 511, part (c) shows the VPTS#B, which 
is a VPTS of the compression video signal B to be 
extended by the video decoder 521, and part (d) shows 
the VPTS#C, which is a VPTS of the compression video 

20 signal C to be extended by the video decoder 531 . 

[0241] The video decoder 511 continues extension 
and reproduction of the compression video signal A, 
and the difference between the VPTS#A and the refer- 
ence time signal exceeds 33 msec, as the threshold 

25 value at T1. Accordingly, the video reproduction timing 
control means of the video decoder 511 skips one 
frame, which is originally to be reproduced, to adjust the 
reproduction timing so that the difference between the 
VPTS#A and the reference time signal is equal to or 

30 less than the threshold value. 

[0242] The video decoder 521 continues extension 
and reproduction of the compression video signal B, 
and the difference between the VPTS#B and the refer- 
ence time signal exceeds -33 msec, as the threshold 

35 value at T2. Accordingly, the video reproduction timing 
control means of the video decoder 521 reproduces one 
frame in repetition, which has been already reproduced, 
to adjust the reproduction timing so that the difference 
between the VPTS#B and the reference time signal is 

40 equal to or less than the threshold value. 

[0243] Similarly, the video decoder 531 continues 
extension and reproduction of the compression video 
signal C, and the difference between the VPTS#C and 
the reference time signal exceeds 33 msec, as the 

45 threshold value at T3. Accordingly, the video reproduc- 
tion timing control means of the video decoder 531 
skips one frame, which is originally to be reproduced, to 
adjust the reproduction timing so that the difference 
between the VPTS#C and the reference time signal is 

50 equal to or less than the threshold value. 

[0244] As described above, in the third example, 
when the difference between the reference time signal 
and the VPTS detected by each video decoder exceeds 
the threshold value, the video reproduction timing con- 

55 trol means of each video decoder performs adjustment 
so that difference between the reference time signal and 
the VPTS does not exceed the threshold value. In this 
manner, the video signals reproduced by video decod- 
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ers can be synchronized with one another. 
(Example 4) 

[0245] The fourth example relates to a reproduction 5 
apparatus for adjusting a reference time signal using 
audio reproduction time information indicating the time 
to reproduce the audio signal and synchronizes a plural- 
ity of video signals based on the reference time signal. 
[0246] Figure 41 shows a data structure of an opti- 10 
cal disk used in an optical disk reproduction apparatus 
in the fourth example. The optical disk includes com- 
pression audio data in addition to the data included in 
the optical disk used in the third example. 
[0247] An audio signal is audio-framed in units of 32 15 
msec, for compression to obtain a compression audio 
stream. The audio stream is packeted in units of 2 kB 
into audio packets and recorded on the optical disk. A 
packet header of each audio packet includes a stream 
ID for indicating that the stored data is a compression 20 
audio stream. When the packet stores a leading part of 
the audio frame, the packet header also includes an 
APTS (audio presentation time stamp) as audio repro- 
duction time information indicating the time to repro- 
duce the frame. 25 
[0248] Figure 39 is a block diagram of the reproduc- 
tion apparatus in the fourth example. 
[0249] Elements 501 through 532 are the same as 
those shown in Figure 35 in the third example. 
[0250] Reference numeral 504 represents a buffer 30 
memory for temporarily storing the compression audio 
signal. Reference numeral 541 represents audio exten- 
sion means for extending the compression audio signal. 
Reference numeral 542 represents a speaker for repro- 
ducing the extended audio signal. 35 
[0251] Figure 40 shows a structure of the audio 
decoder 541. The audio decoder 541 includes APTS 
detection means 701 for detecting the APTS stored in a 
packet header of the audio packet, and audio extension 
means 702 for extending the compression audio 40 
stream. 

[0252] An operation of the optical disk reproduction 
apparatus shown in Figure 39 for reproducing the opti- 
cal disk shown in Figure 41 will be described. 
[0253] The operation until the signal is input to the 45 
division means 505 is similar to that with the optical disk 
reproduction apparatus in the third example. 
[0254] The data read from the buffer memory 504 is 
divided into compression video signals A through C and 
a compression audio signal by the division means 505 50 
and output. The division means 505 identifies which of 
the compression video signals A through C and the 
compression audio signal is stored in each packet with 
the stream ID in the packet header of the packeted data, 
and determines the destination based on the identifica- 55 
tion result. 

[0255] The divided compression video signals and 
compression audio signal are temporarily stored in 
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buffer memories 510 through 540 respectively. 
[0256] The video decoders 511 through 531 read 
data from the buffet memories 510 through 530 respec- 
tively, extend the compression video signals, and output 
the signals as video signals to the monitors 512 through 
532 respectively. The audio decoder 541 reads data 
from the buffer memory 540, extends the compression 
audio signal, and outputs the signal as an audio signal 
through the speaker 542. 

[0257] The operations of the video decoders 511 
through 531 for extending the compression video sig- 
nals and for adjusting the synchronization when the dif- 
ference between the reference time signal and the 
VPTS exceeds the threshold value are the same as in 
the third example. 

[0258] The compression audio signal read from the 
buffer memory 540 is input to the audio decoder 541. 
The APTS detection means 701 detects and outputs the 
APTS. The audio extension means 702 extends the 
compression audio stream and outputs the audio signal. 
[0259] The APTS signal output from the audio 
decoder 541 is input to the reference time signal gener- 
ation means 506, and the reference time signal is 
adjusted by the APTS. 

[0260] In the fourth example, due to the precision 
error of the crystal oscillator used in the reference time 
signal generation means 506, the video decoders 511 
through 531 and the audio decoder 541, the reference 
time signal is faster in terms of extension and reproduc- 
tion relative to the audio decoder 541. The video 
decoder 511 is slower and the video decoder 521 is 
faster in terms of extension and reproduction relative to 
the reference time signal. Unless the reproduction tim- 
ing is adjusted, the reproduced video signals and audio 
signal are out of synchronization. 
[0261 ] Figure 42 is a timing diagram of audio repro- 
duction in the fourth example. Part (a) of Figure 42 
shows the APTS with respect to reproduction time t. 
Part (b) shows the reference time signal. Part (c) shows 
the VPTS#A, at which the compression video signal A 
to be extended by the video decoder 51 1 is to be repro- 
duced, and part (d) shows the VPTS#B, at which the 
compression video signal B to be extended by the video 
decoder 512 is reproduced. 

[0262] Figure 42 does not show the VPTS#C, at 
which the compression video signal C to be extended by 
the video decoder 531, but the diagram is almost the 
same as in Figure 38 shown regarding the third exam- 
ple. 

[0263] The reference time signal generation means 
506 is adjusted using the APTS at time when the APTS 
shows ta1 and ta2, and the reference time signal is 
reset as ta1 and ta2 at the respective time. 
[0264] The video decoder 511 continues extension 
and reproduction of the compression video signal A, 
and the difference between the VPTS#A and the refer- 
ence time signal exceeds 33 msec, as the threshold 
value at T4. Accordingly, the video reproduction timing 
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control means of the video decoder 511 skips one 
frame, which is originally to be reproduced, to adjust the 
reproduction timing so that the difference between the 
VPTS#A and the reference time signal is equal to or 
less than the threshold value. 5 
[0265] The video decoder 521 continues extension 
and reproduction of the compression video signal B, 
and the difference between the VPTS#B and the refer- 
ence time signal exceeds -33 msec, as the threshold 
value at T5 and T6. Accordingly, the video reproduction 10 
timing control means of the video decoder 521 repro- 
duces one frame in repetition, which has been already 
reproduced, to adjust the reproduction timing so that the 
difference between the VPTS#B and the reference time 
signal is equal to or less than the threshold value. 15 
[0266] As described above, in the fourth example, 
when the difference between the reference time signal 
and the VPTS detected by each video decoder exceeds 
the threshold value, the video reproduction timing con- 
trol means of each video decoder performs adjustment 20 
so that difference between the reference time signal and 
the VPTS does not exceed the threshold value. In this 
manner, the video signals reproduced by video decod- 
ers can be synchronized with one another. 
[0267] Regarding the difference between the refer- 25 
ence time signal and the APTS, the APTS is not 
adjusted using the reference time signal but the refer- 
ence time signal is adjusted using the APTS. Accord- 
ingly, audio and video signals are synchronized with no 
unnaturalness in the audio output. 30 

(Example 5) 

[0268] The fifth example relates to a reproduction 
apparatus for adjusting the reference time signal using a 35 
VPTS detected by one video decoder and synchroniz- 
ing a plurality of video signals based on the reference 
time signal. 

[0269] Figure 43 is a block diagram of an optical 
disk reproduction apparatus in the fifth example. 40 
[0270] Elements 501 through 532 are the same as 
those in the third example. Reference numeral 551 rep- 
resents a video decoder used in the fifth example. 
[0271 ] The video decoder 551 has a function of out- 
putting the detected VPTS. Figure 44 shows a structure 45 
of the video decoder 551 . 

[0272] The video decoder 551 includes VPTS 
detection means 801 for detecting a VPTS indicating 
the reproduction time of the video signal multiplexed as 
the compression video signal and video extension so 
means 802 for extending the compression video signal. 
[0273] In the fifth example, due to the precision 
error of the crystal oscillator used in the reference time 
signal generation means 506 and the video decoders 
521, 531 and 551, the reference time signal is faster in 55 
terms of extension and reproduction relative to the 
video decoder 551. The video decoder 521 is slower 
and the video decoder 531 is faster in terms of exten- 



sion and reproduction relative to the reference time sig- 
nal. Unless reproduction timing is adjusted, the 
reproduced video signals are out of synchronization. 
[0274] Figure 45 is a timing diagram of video output 
in the fifth example. Part (a) of Figure 45 shows the 
VPTS#A detected by the video decoder 551 with 
respect to reproduction time t. Part (b) shows the refer- 
ence time signal. Part (c) shows VPTS#B, at which the 
compression video signal B to be extended by the video 
decoder 521 is to be reproduced, and part (d) shows the 
VPTS#C, at which the compression video signal C to be 
extended by the video decoder 531 is to be reproduced. 
[0275] The reference time signal generation means 
506 is adjusted using the VPTS#A at time when the 
VPTS#A shows tv1 and tv2, and the reference time sig- 
nal is reset as tv1 and tv2 at the respective time. 
[0276] The video decoder 521 continues extension 
and reproduction of the compression video signal B, 
and the difference between the VPTS#B and the refer- 
ence time signal exceeds 33 msec, as the threshold 
value at T7. Accordingly, the video reproduction timing 
control means of the video decoder 521 skips one 
frame, which is originally to be reproduced, to adjust the 
reproduction timing so that the difference between the 
VPTS#B and the reference time signal is equal to or 
less than the threshold value. 

[0277] Similarly, the video decoder 531 continues 
extension and reproduction of the compression video 
signal C, and the difference between the VPTS#C and 
the reference time signal exceeds -33 msec, as the 
threshold value at T8 and T9. Accordingly, the video 
reproduction timing control means of the video decoder 
531 reproduces one frame in repetition, which has been 
already reproduced, to adjust the reproduction timing so 
that the difference between the VPTS#C and the refer- 
ence time signal is equal to or less than the threshold 
value. 

[0278] As described above, in the fifth example, 
when the difference between the reference time signal 
and the values of VPTSs detected by the video decod- 
ers 521 and 531 exceeds the threshold value, the video 
reproduction timing control means of each video 
decoder performs adjustment so that the difference 
between the reference time signal and the VPTS does 
not exceed the threshold value. 

[0279] By adjusting the reference time signal using 
the VPTS#A detected by the video decoder 551, the 
video signal reproduced by the video decoder 551 is not 
accompanied by any unnaturalness in the visual output 
despite the frame-by-frame skipping or repeat of the 
reproduction. Thus, the video signals can be synchro- 
nized with one another. 

(Example 6) 

[0280] The sixth example relates to a reproduction 
apparatus including a plurality of video decoders for 
extending and reproducing a compression video signal. 
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Each of the video decoders includes reference time sig- 
nal generation means. The reproduction apparatus 
adjusts the reference time signal of each video decoder 
using an APTS indicating the time to reproduce an 
audio signal to realize synchronization. 
[0281] In the sixth example, the optical disk shown 
in Figure 41 is used. 

[0282] Figure 46 is a block diagram of an optical 
disk reproduction apparatus in the sixth example. 
[0283] Elements 501 through 542 are the same as 
those shown in Figure 39 in the fourth example. Unlike 
the reproduction apparatus shown in Figure 39, the 
reproduction apparatus in this example does not have 
reference time signal generation means 506 independ- 
ently, but each video decoder 561 through 581 has ref- 
erence time signal generation means. 
[0284] Reference numeral 561 represents a video 
decoder for extending and reproducing compression 
video signal A, reference numeral 571 represents a 
video decoder for extending and reproducing compres- 
sion video signal B, and reference numeral 581 repre- 
sents a video decoder for extending and reproducing 
compression video signal C. 

[0285] Figure 47 shows a structure of each of the 
video decoders 561 through 581 used in the sixth exam- 
ple. 

[0286] The video decoder includes VPTS detection 
means 901 for detecting a VPTS indicating the repro- 
duction time of the video signal multiplexed as the com- 
pression video signal, video extension means 902 for 
extending the compression video signal, and video 
reproduction timing control means 903 for comparing 
the reference time signal and the VPTS and skipping or 
repeating the video reproduction on a frame-by-frame 
basis when the comparison result exceeds the thresh- 
old value, and reference time signal generation means 
904 for generating the reference time signal. 
[0287] In the sixth example, the reference time sig- 
nal of reference time signal generation means 904 
included in each of the video decoders 561 through 581 
is adjusted using the APTS detected by the audio 
decoder 541 . 

[0288] Since the reference time signals are 
adjusted using the same APTS, the reference time sig- 
nals generated in the video decoders 561 through 581 
show the same value after being adjusted. 
[0289] After the adjustment using the APTS, as in 
the fourth example, when the difference between the 
reference time signal and the values of VPTS detected 
by each video decoder exceeds the threshold value, the 
video reproduction timing control means of each video 
decoder performs adjustment by skipping or repeating 
the reproduction on a frame-by-frame basis so that dif- 
ference between the reference time signal and the 
VPTS does not exceed the threshold value. 
[0290] As described above, in the sixth example, 
the reference time signal generated in each video 
decoder is adjusted using an APTS, and the video 



reproduction timing control means of each video 
decoder maintains the difference between each refer- 
ence time signal and each VPTS to be equal to or less 
than the threshold value. Thus, the video signals can be 

5 synchronized with one another. 

[0291] As in the fourth example, the audio signal 
and the video signal can be synchronized without pro- 
viding any unnaturalness in the audio output. 
[0292] In the sixth example, the reference time sig- 

10 nals in the video decoders 561 through 581 are 
adjusted using the APTS detected by the audio decoder 
541 . The video signals can be reproduced in synchroni- 
zation in a similar manner by using one of the video 
decoders shown in Figure 44 in the fifth example and 

15 adjusting the reference time signals of the other video 
decoders using the VPTS detected by the one video 
decoder. 

(Example 7) 

20 

[0293] The seventh example relates to a reproduc- 
tion apparatus for simultaneously reproducing two com- 
pression video signals. The two compression video 
signals are obtained by dividing a 3D signal into a right- 

25 eye video signal and a left-eye video signal and com- 
pressing the divided video signals. 
[0294] The overall structure of the apparatus is gen- 
erally similar to that of the optical disk reproduction 
apparatus shown in Figure 46 in the sixth example, but 

30 the reproduction apparatus in the seventh example 
includes two video decoders for extending compression 
video signals obtained after the division means 505 
since two video signals are to be reproduced simultane- 
ously. Figure 48 shows a structure of one of the video 

35 decoders used in the seventh example, and Figure 49 
shows a structure of the other video decoder used in the 
seventh example. 

[0295] As shown in Figure 48, the video decoder 
includes VPTS detection means 1001 for detecting a 

40 VPTS indicating the reproduction time of the video sig- 
nal multiplexed as the compression video signal, video 
extension means 1002 for extending the MPEG com- 
pression video signal, reference time signal generation 
means 1004 for generating a reference time signal, and 

45 video reproduction timing control means 1003 for com- 
paring the reference time signal and the VPTS and skip- 
ping or repeating the video reproduction on a frame-by- 
frame basis when the comparison result exceeds the 
threshold value and also for outputting a horizontal syn- 

50 chronization signal and a vertical synchronization signal 
for the video to be reproduced. 

[0296] As shown in Figure 49, the other video 
decoder includes VPTS detection means 1101 for 
detecting a VPTS indicating the reproduction time of the 
55 video signal multiplexed as the compression video sig- 
nal, video extension means 1102 for extending the 
MPEG compression video signal, reference time signal 
generation means 1104 for generating a reference time 
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signal, and video reproduction timing control means 
1103 for comparing the reference time signal and the 
VPTS and skipping or repeating the video reproduction 
on a frame-by-frame basis when the comparison result 
exceeds the threshold value, receiving the horizontal 
synchronization signal and the vertical synchronization 
signal for the video to be reproduced, and also repro- 
ducing the extended video signal in synchronization 
with the horizontal/vertical synchronization signals. 
[0297] The video decoders are connected to each 
other so that the horizontal synchronization signal and 
the vertical synchronization signal output from the video 
decoder in Figure 48 are sent to the video decoder in 
Figure 49. 

[0298] In the optical disk reproduction apparatus in 
the seventh example having the above-described struc- 
ture, the reference time signal generated by each video 
decoder for the right or left eye is adjusted using an 
APTS, and the video reproduction timing control means 
of each video decoder maintains the difference between 
each reference time signal and each VPTS to be equal 
to or less than the threshold value. Thus, the right-eye 
video signal and the left-eye video signal can be syn- 
chronized with one another on a frame-by-frame basis. 
By using the horizontal and vertical synchronization sig- 
nals output by one of the video decoder as the horizon- 
tal and the vertical synchronization signals of the other 
video decoder, two video signals can be reproduced in 
synchronization on a pixel-by-pixel basis. 
[0299] In the seventh example, compression video 
signals obtained from a 3D video signal are used and 
divided into the right-eye and left-eye video signals. 
Alternatively, for example, an original video signal hav- 
ing a first resolution is divided in a vertical and/or hori- 
zontal direction into at least two video signals including 
a first video signal and a second video signal having a 
second resolution which is lower than the first resolu- 
tion. The resultant signals are compressed to be used. 
Thus, a plurality of video signals in synchronization with 
one another on a pixel-by-pixel basis can be obtained 
as from a 3D video signal. By synthesizing such result- 
ant signals, the clear original video signal having the 
original resolution is reproduced. 

(Example 8) 

[0300] The eighth example relates to an optical disk 
reproduction apparatus for extending one compression 
video signal and two compression audio signals and 
reproducing the signals simultaneously. 
[0301 ] Figure 52 shows a data structure of the opti- 
cal disk used in the eighth example. 
[0302] Two audio signals D and E are compressed 
to obtain compression audio streams D and E. A video 
signal is compressed to obtain a compression video 
stream. 

[0303] The compression video streams D and E 
and the compression video stream are packeted in units 



of 2 kB into audio packets and video packets. A packet 
header of each packet includes a stream ID for indicat- 
ing which of the compression audio streams D and E 
and the compression video stream is stored, and the 

5 APTS and VPTS described above. 

[0304] Figure 50 is a block diagram of a reproduc- 
tion apparatus in the eighth example. 
[0305] The reproduction apparatus has a generally 
similar structure to that in Figure 39 in the fourth exam- 

w pie. The audio decoder 541 has the same structure as 
that shown in Figure 40, and the video decoder 531 has 
the same structure as that shown in Figure 36. The 
audio decoder 591 has the same structure as that 
shown in Figure 51. 

15 [0306] Reference numeral 590 represents a buffer 
memory for temporarily storing the compression audio 
signal like the buffer memory 540. Reference numeral 
592 represents a speaker for reproducing the audio sig- 
nal. 

20 [0307] Figure 51 shows a structure of the audio 
decoder 591 . 

[0308] The audio decoder 591 includes APTS 
detection means 1201 for detecting an APTS of an 
audio signal multiplexed as a compression audio signal, 

25 audio extension means 1202 for extending the com- 
pression audio signal, and audio reproduction timing 
control means 1203 for comparing the reference time 
signal and the APTS and skipping or repeating the 
audio reproduction on an audio frame-by-audio frame 

30 basis when the comparison result exceeds the thresh- 
old value. 

[0309] A reproduction operation in the eighth exam- 
ple will be described. 

[0310] The operation until the signal read from the 
35 optical disk 501 is input to the division means 505 is 
similar to that in the other examples. 
[031 1] The data read from the buffer memory 504 is 
divided by the division means 505 into a compression 
video signal, the compression audio signal D and the 
40 compression audio signal E, and output. The division 
means 505 identifies which of the compression video 
signal, the compression audio signal D and the com- 
pression audio signal E is stored in each packet using 
the packet ID in the packet header of the packeted data, 
45 and determines the destination based on the identifica- 
tion result. 

[0312] The divided compression video signal, the 
compression audio signal D and compression audio sig- 
nal E are temporarily stored in buffer memories 530, 

50 540 and 590 respectively. 

[0313] The video decoders reads data from the 
buffer memory 530, extends the compression video sig- 
nal and outputs the signal as a video signal to a monitor 
532. The audio decoders 541 and 591 read data from 

55 the buffer memories 540 and 590, extend the compres- 
sion audio signals and output the signals as audio sig- 
nals through the speakers 542 and 592. 
[0314] The reference time signal generated by the 
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reference time signal generation means 506 is adjusted 
by an APTS#D detected by the audio decoder 541 . 
[0315] In the audio decoder 591, an APTS#E is 
detected by the APTS detection means 1201 and the 
compression audio signal E is extended by the audio 
extension means 1202. The audio reproduction timing 
control means 1203 receives the extended audio signal 
output from the audio extension means 1202, the refer- 
ence time signal, and the APTS#E from the APTS 
detection means 1201, and compares the reference 
time signal and the APTS#E. When the difference 
between the reference time signal and the APTS#E 
exceeds the threshold value, the audio reproduction 
timing control means 1203 controls the audio reproduc- 
tion timing so that the difference is equal to or less than 
the threshold value. 

[0316] In the eighth example, 32 msec is used as 
the threshold value. The audio reproduction timing con- 
trol means 1203 performs the following. 

(reference time signal-APTS#E)>32 msec.:1 audio 
frame is skipped. 

(reference time signal-APTS#E)<-32 msec.:1 audio 
frame is repeated. 

[0317] The operation of the video decoder 531 for 
extending the compression video signal and performing 
adjustment when the difference between the reference 
time signal and the VPTS exceeds the threshold value 
are similar to those in the third example. 
[0318] In the eighth example, due to the precision 
error of the crystal oscillator used in the reference time 
signal generation means 506, the video decoder 531, 
and the audio decoders 541 and 591 ; the audio decod- 
ers 541 and 591 are slower and the video decoder 531 
is faster in terms of extension and reproduction relative 
to the reference time signal. Unless reproduction timing 
is adjusted, the reproduced video signals are out of syn- 
chronization. 

[0319] Figure 53 is a timing diagram of video repro- 
duction in the eighth example. Part (a) of Figure 53 
shows the APTS#D with respect to reproduction time t. 
Part (b) shows the reference time signal, part (c) shows 
APTS#E, at which the compression audio signal E to be 
extended by the audio decoder 591 is to be reproduced, 
and part (d) shows the VPTS, at the compression video 
signal to be extended by the video decoder 531 is to be 
reproduced. The reference time signal is adjusted using 
the APTS#D when APTS#D shows ta3 and ta4. The ref- 
erence time signal is reset to ta3 and ta4 at the respec- 
tive time. 

[0320] The audio decoder 591 continues extension 
and reproduction of the compression audio signal E, 
and the difference between the APTS#E and the refer- 
ence time signal exceeds 32 msec, as the threshold 
value at T10. Accordingly, the audio reproduction timing 
control means 1203 of the audio decoder 591 skips one 
audio frame, which is originally to be reproduced, to 



adjust the reproduction timing so that the difference 
between the APTS#E and the reference time signal is 
equal to or less than the threshold value. 
[0321] The difference between the VPTS and the 

5 reference time signal exceeds -33 msec, as the thresh- 
old value at T1 1 and T12. Accordingly, the video repro- 
duction timing control means of the video decoder 531 
reproduces one frame in repetition, which has been 
already reproduced at the respective time, to adjust the 

w reproduction timing so that the difference between the 
VPTS and the reference time signal is equal to or less 
than the threshold value. 

[0322] As described above, in the eighth example, 
when the difference between the reference time signal 

15 and the APTS#E detected by the audio decoder 591 
exceeds the threshold value, the audio reproduction 
timing control means of the audio decoder performs 
adjustment so that difference between the reference 
time signal and the APTS#E does not exceed the 

20 threshold value of audio reproduction. Similarly, differ- 
ence between the reference time signal and the VPTS 
is adjusted so as not to exceed the threshold value of 
video reproduction. In this manner, each audio signal 
and the video signal can be synchronized with one 

25 another. 

(Example 9) 

[0323] In the ninth example, the clock for perform- 
30 ing extension is changed for audio reproduction timing 
control. 

[0324] The overall structure and operation of the 
reproduction apparatus in the ninth example are gener- 
ally similar to those of the optical disk reproduction 
35 apparatus in the eighth example, but the operation of 
audio reproduction timing control performed when the 
reference time signal and the APTS#E exceeds the 
threshold value is different from that of the eighth exam- 
ple. With reference to Figure 54 and 55, audio reproduc- 
40 tion timing control used in the ninth example will be 
described. 

[0325] Figure 54 shows an operation when the dif- 
ference between the APTS#E and the reference timing 
signal exceeds 32 msec, which is the threshold for the 
45 audio reproduction. Part (a) of Figure 54 shows the ref- 
erence time signal with respect to reproduction time t. 
Part (b) shows the APTS#E, and part (c) shows the 
clock frequency at which the audio decoder 591 per- 
forms extension and reproduction. Ordinary extension 
50 and reproduction are performed by clock fO having a fre- 
quency which is 384 times the sampling frequency fs of 
the audio signal. The difference between the APTS#E 
and the reference time signal exceeds 32 msec, at time 
T1 1 , and accordingly, audio reproduction control means 
55 switches the clock fO to f1 . The frequency of clock f1 is 
higher by 1 0% than the frequency of clock fO. Extension 
performed with clock f1 proceeds faster than extension 
performed with clock fO by 1 0%. With clock f1 , the 
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extension is performed for 320 msec, from the point 
where the difference between the APTS#E and the ref- 
erence time signal exceeds 32 msec, which is the 
threshold value. Thus, the reproduction timing is 
adjusted so that the difference between the APTS#E 5 
and the reference time signal is equal to or less than the 
threshold value. 

[0326] Figure 55 shows an operation when the dif- 
ference between the APTS#E and the reference timing 
signal exceeds -32 msec, which is the threshold for the 10 
audio reproduction. Part (a) of Figure 55 shows the ref- 
erence time signal with respect to reproduction time t. 
Part (b) shows the APTS#E, and part (c) shows the 
clock frequency at which the audio decoder 591 per- 
forms extension and reproduction. 15 
[0327] The difference between the APTS#E and the 
reference time signal exceeds -32 msec, at time T12, 
and accordingly, audio reproduction control means 
switches the clock fO to f2. The frequency of clock f2 is 
lower by 1 0% than the frequency of clock fO. Extension 20 
performed with clock f2 proceeds more slowly than 
extension performed with clock fO by 1 0%. With clock f2, 
the extension is performed for 320 msec, from the point 
where the difference between the APTS#E and the ref- 
erence time signal exceeds -32 msec, which is the 25 
threshold value. Thus, the reproduction timing is 
adjusted so that the difference between the APTS#E 
and the reference time signal is equal to or less than the 
threshold value. 

[0328] As described above, when the difference 30 
between the APTS#E and the reference time signal 
exceeds the threshold value for the audio reproduction, 
the clock by which the signal is extended is changed so 
that the extension is performed at a higher speed or 
lower speed than the normal speed. By such an opera- 35 
tion, the reproduction timing is controlled so that the dif- 
ference between the APTS#E and the reference time 
signal is equal to or less than the threshold value. Thus, 
the audio signals and the video signal can be repro- 
duced in synchronization with no unnaturalness in the 40 
audio output. 

[0329] In the ninth example, the frequency of the 
clock is changed by 10%. Needless to say, a more nat- 
ural audio signal is obtained by changing the clock less 
or gradually. 45 
[0330] In the eighth and ninth examples, the refer- 
ence time signal is adjusted using the APTS#D. Alterna- 
tively, the video decoder shown in Figure 44 can be 
used, in which case the VPTS output from the video 
decoder can be used for adjustment. so 
[0331] The present invention has been described 
by way of specific examples. 

[0332] The comparison between the reference time 
signal and the VPTS or APTS, control of the reproduc- 
tion time, adjustment of the reproduction timing using a 55 
VPTS or APTS can be performed by a microcomputer 
which controls the entirety of the reproduction appara- 
tus. 
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[0333] In the above examples, the present invention 
is applied to optical disk reproduction apparatuses. The 
virtual frame-based editing system according to the 
present invention is also applicable to a reproduction 
apparatus, referred to as the set top box, for extending 
compression signals supplied through communication 
networks or digital satellite broadcasting. Even when 
the programs are switched, the non-continuous video 
signals are connected seamlessly, which provides a sig- 
nificant advantage. 

INDUSTRIAL APPLICABILITY 

[0334] A basic video signal and an interpolation sig- 
nal are divided into frames each having 1 GOP or more 
and subjected to interleaving alternately to be recorded 
on the optical disk as interleave blocks. From such an 
optical disk, a high resolution synthesis reproduction 
apparatus reproduces information in both of two types 
of interleave blocks alternately arranged. When the opti- 
cal disk having a high resolution video signal is repro- 
duced by a non-progressive reproduction apparatus, 
information in the interleave block of only odd field or 
even field is reproduced by track jump. Thus, a com- 
plete two-dimensional video is obtained. Thus, compat- 
ibility is realized. 

[0335] Especially, a high resolution video signal 
arrangement information file and a high resolution pic- 
ture identifier are recorded on the optical disk. Accord- 
ingly, the location of the high resolution video signal is 
easily determined. Therefore, two ordinary interlace sig- 
nals can be made into a progressive signal. In addition, 
output of different content pictures for the right eye and 
left eye can be avoided. 

[0336] According to the two-stream simultaneous 
reproduction synchronization system according to the 
present invention, an MPEG signal can be virtually 
edited on a frame-by-frame basis, which is convention- 
ally performed only on a GOP-by-GOP basis if deterio- 
ration of picture quality should be avoided. By recording 
reproduction control information, signals can be con- 
nected on a frame-by-frame basis when being repro- 
duced. Thus, virtual frame-based editing can be 
realized without deterioration in the picture quality. 
[0337] In the two-stream simultaneous reproduction 
synchronization system, a plurality of compression 
video signals or a plurality of compression audio signals 
can be reproduced in synchronization with one another 
after being extended. 

[0338] In a reproduction apparatus in which the ref- 
erence time signal is adjusted using an APTS detected 
by an audio decoder and the video output timing is con- 
trolled so that the VPTS matches the adjusted reference 
time signal, an audio signal and a plurality of video sig- 
nals are synchronized for reproduction with no unnatu- 
ralness in the audio output. 

[0339] In a reproduction apparatus in which the 
audio output timing is controlled by changing an exten- 
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sion clock, audio and video signals are synchronized for 
reproduction with no unnaturalness in the audio output 
with no interruption or pause in the audio signal. 

Claims s 

1. An optical disk reproduction apparatus for repro- 
ducing a signal recorded on an optical disk, in 
which the optical disk has, recorded thereon, at 
least a first video stream representing a low fre- 10 
quency component of the video signal and a sec- 
ond video stream representing at least a high 
frequency component of the video signal, the first 
video stream includes a plurality of first interleave 
units and the second video stream includes a plu- 15 
rality of second interleave units, each of the plurality 
of first interleave units includes ml GOPs (where 
ml is an integer of 1 or greater), each of the plural- 
ity of second interleave units includes m2 GOPs 
(where m2 is an integer of 1 or greater), the optical 20 
disk reproduction apparatus comprising: 



2. An optical disk reproduction apparatus according to 
claim 1 , wherein the plurality of first interleave units 
are each corresponded to first time information 
relating to reproduction time, and the plurality of so 
second interleave units are each corresponded to 
second time information relating to reproduction 
time. 

3. An optical disk reproduction apparatus according to 55 
claim 2, further comprising: 

a reference time signal generation section for 



generating a reference time signal; 
a first reproduction control section for control- 
ling the reproduction time of the first reproduc- 
tion signal in accordance with the difference 
between the reference time signal and the first 
time information; 

a second reproduction control section for con- 
trolling the reproduction time of the second 
reproduction signal in accordance with the dif- 
ference between the reference time signal and 
the second time information; and 
an adjusting section for adjusting the reference 
time signal so that the reference time signal 
supplied to the first reproduction control section 
and the reference signal supplied to the second 
reproduction control section represent sub- 
stantially the same time. 

4. An optical disk reproduction apparatus according to 
claim 3, wherein the adjusting section adjusts the 
reference time signal based on audio reproduction 
time information representing the time to reproduce 
an audio signal which is to be output in synchroni- 
zation with the video signal. 

5. An optical disk reproduction apparatus according to 
claim 3, wherein, the adjusting section adjusts the 
reference time signal based on at least one of first 
video reproduction time information representing 
the time to reproduce the first reproduction signal 
and second video reproduction time information 
representing the time to reproduce the second 
reproduction signal. 

6. An optical disk reproduction apparatus according to 
claim 3, wherein: 

the first reproduction control section controls 
the reproduction time of the first reproduction 
signal by skipping a frame of the first reproduc- 
tion signal or by reproducing a frame of the first 
reproduction signal in repetition; and 
the second reproduction control section con- 
trols the reproduction time of the second repro- 
duction signal by skipping a frame of the 
second reproduction signal or by reproducing a 
frame of the second reproduction signal in rep- 
etition. 

7. An optical disk reproduction apparatus according to 
claim 1, wherein at least one of the first time infor- 
mation and the second time information includes at 
least one of a PTS, a DTS and an SCR. 

8. An optical disk reproduction apparatus according to 
claim 1 , wherein: 

the first reproduction signal corresponds to a 



a reproduction section for reproducing the first 
video stream and the second video stream 
recorded on the optical disk; 25 
a division section for dividing the reproduced 
first video stream into the plurality of first inter- 
leave units and for dividing the reproduced sec- 
ond video stream into the plurality of second 
interleave units; 30 
a decoding section for decoding the plurality of 
first interleave units to generate a first repro- 
duction signal representing the low frequency 
component of the video signal and for decoding 
the plurality of second interleave units to gener- 35 
ate a second reproduction signal representing 
at least the high frequency component of the 
video signal; 

a synthesis section for synthesizing the first 
reproduction signal and the second reproduc- 40 
tion signal to generate the video signal; and 
an output section for selectively outputting at 
least one of the first reproduction signal, the 
second reproduction signal, and the video sig- 
nal. 45 
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first pixel number, and the second reproduction 
signal corresponds to a second pixel number, 
which is larger than the first pixel number, 
the synthesis section includes a converter for 
converting the first reproduction signal into a 5 
conversion signal corresponding to the second 
pixel number, and 

the video signal is obtained by synthesizing the 
conversion signal and the second reproduction 
signal. 10 



9. An optical disk reproduction apparatus according to 
claim 8, wherein the optical disk further has, 
recorded thereon, an identifier representing the first 
pixel number corresponding to the first reproduction 15 
signal, and the converter converts the first repro- 
duction signal into the conversion signal in accord- 
ance with the identifier. 

10. An optical disk reproduction apparatus according to 20 
claim 8, wherein: 

the optical disk further has, recorded thereon, 
an identifier representing the first pixel number 
corresponding to the first reproduction signal, 25 
and 

the optical disk reproduction apparatus further 
includes a rotation control section for control- 
ling the rotation of the optical disk, and 
the rotation control section controls the rotation 30 
of the optical disk in accordance with the iden- 
tifier. 

1 1 . An optical disk reproduction apparatus according to 
claim 1 , wherein: 35 



12. An optical disk reproduction apparatus according to 
claim 11, further comprising a buffer memory sec- 
tion for storing the plurality of first interleave units 
and the plurality of second interleave units, and the 
buffer memory section has a capacity which is 
equal to or greater than an amount of data of the 
GOP or GOPs included in the second interleave 
units. 

13. An optical disk reproduction apparatus according to 



claim 12, wherein the buffer memory section has a 
capacity which is 1 MB or greater. 

14. An optical disk including, recorded thereon, at least 
a first video stream representing a low frequency 
component of the video signal and a second video 
stream representing at least a high frequency com- 
ponent of the video signal, wherein: the first video 
stream includes a plurality of first interleave units, 
the second video stream includes a plurality of sec- 
ond interleave units, each of the plurality of first 
interleave units includes ml GOPs (where ml is an 
integer of 1 or greater), and each of the plurality of 
second interleave units includes m2 GOPs (where 
m2 is an integer of 1 or greater). 

15. An optical disk according to claim 14, wherein the 
plurality of first interleave units and the plurality of 
second interleave units are structured so that repro- 
duction time of one of the plurality of first interleave 
units is substantially equal to reproduction time of 
one of the plurality of second interleave units, the 
one of the plurality of second interleave units corre- 
sponding to the one of the plurality of first interleave 
units. 

16. An optical disk recording apparatus, comprising: 

a dividing section for dividing a video signal into 
a first video signal representing a low fre- 
quency component of the video signal and a 
second video signal representing at least a 
high frequency component of the video sig- 
nal;an encoding section for generating a first 
video stream by encoding the first video signal 
and for generating a second video stream by 
encoding the second video signal, wherein: the 
first video stream includes a plurality of first 
interleave units, the second video stream 
includes a plurality of second interleave units, 
each of the plurality of first interleave units 
includes ml GOPs (where ml is an integer of 1 
or greater), and each of the plurality of second 
interleave units includes m2 GOPs (where m2 
is an integer of 1 or greater); 
a selection output section for selectively output- 
ting the plurality of first interleave units included 
in the first video stream and the plurality of sec- 
ond interleave units included in the second 
video stream; and 

a recording section for recording the signal out- 
put from the selection output section on an opti- 
cal disk. 

17. An optical recording apparatus according to claim 
16, wherein the division section includes a decoder 
for decoding the first video stream and a differential 
calculator for calculating a differential between the 



the optical disk further has, recorded thereon, 
an identifier representing that the video signal 
is obtained by encoding a progressive video 
signal of 24 frames to 30 frames per second, 40 
the output section includes a converter for con- 
verting at least one of the first reproduction sig- 
nal, the second reproduction signal, and the 
video signal into a frame signal, and 
the output section outputs the progressive 45 
video signal of 60 frames per second by output- 
ting the frame signal in an overlapping manner. 
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video signal and the signal output from the decoder, 
and outputs the signal output from the differential 
calculator as the second video signal. 

18. An optical recording apparatus according to claim 5 
17, 

wherein the division section further includes: 

a first converter for converting the video signal 
into a first conversion signal corresponding to a 10 
second pixel number which is smaller than a 
first pixel number corresponding to the video 
signal, and 

a second converter for converting the signal 
output from the decoder into a second conver- 15 
sion signal corresponding to the first pixel 
number which is larger than the second pixel 
number corresponding to the signal output 
from the decoder, 

wherein, the division section outputs the first 20 
conversion signal as the first video signal, and 
wherein the differential calculator calculates 
the differential between the video signal and 
the second conversion signal. 

25 

19. An optical disk recording apparatus according to 
claim 17, wherein the recording section further 
records on the optical disk an identifier representing 
that the second video signal is output from the dif- 
ferential calculator. 30 

20. An optical disk recording apparatus according to 
claim 17, wherein the recording section further 
records on the optical disk an identifier representing 

the first pixel number corresponding to the video 35 
signal. 

21. An optical disk recording apparatus according to 
claim 17, wherein the recording section further 
records on the optical disk an identifier representing 40 
the second pixel number corresponding to the first 
video signal. 

22. An optical disk recording apparatus, comprising: 

45 

an input section for receiving an encoded first 
video stream corresponding to a first pixel 
number and an encoded second video stream 
corresponding to a second pixel number which 
is different from the first pixel number, wherein 50 
the first video stream includes a plurality of first 
interleave units, the second video stream 
includes a plurality of second interleave units, 
each of the plurality of first interleave units 
includes ml GOPs (where ml is an integer of 1 55 
or greater), and each of the plurality of second 
interleave units includes m2 GOPs (where m2 
is an integer of 1 or greater); 



a selection output section for selectively output- 
ting the plurality of first interleave units included 
in the first video stream and the plurality of sec- 
ond interleave units included in the second 
video stream; and 

a recording section for recording the signal out- 
put from the selection output section on an opti- 
cal disk. 

23. An optical disk reproduction apparatus for repro- 
ducing a signal recorded on an optical disk, in 
which the optical disk has, recorded thereon, at 
least a first video stream including a plurality of first 
GOPs and a second video stream including a plu- 
rality of second GOPs, each of the plurality of first 
GOPs includes a plurality of pictures, and each of 
the plurality of second GOPs includes a plurality of 
pictures, the optical disk reproduction apparatus 
comprising: 

a reproduction section for reproducing the first 
video stream and the second video stream 
recorded on the optical disk; 
a decoding section for decoding the first video 
stream and the second video stream; and 
an output section for selectively outputting the 
decoded first video stream and the decoded 
second video stream in accordance with repro- 
duction control information, 
wherein the reproduction control information 
indicates that after a first picture included in a 
final first GOP among the plurality of first GOPs 
included in the first video stream is reproduced, 
a second picture included in a leading second 
GOP among the plurality of second GOPs 
included in the second video stream is repro- 
duced, the second picture being different from 
a leading picture of the leading second GOP. 

24. An optical disk reproduction apparatus according to 
claim 23, wherein the decoding section starts 
decoding the second video stream so that the 
decoding of the second picture has been completed 
when the reproduction of the first picture is com- 
pleted. 

25. An optical disk reproduction apparatus according to 
claim 24, wherein: 

the reproduction control information includes 
information ts1 representing a position of the 
first picture, information ts2 representing a 
position of the second picture, and information 
tsG representing a position of the leading pic- 
ture of the leading second GOP, and 
the decoding section finds a decoding start 
position ta in accordance with expression 
ta=ts1 -(ts2-tsG) , and starts decoding the sec- 
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ond video stream based on the decoding start 
position ta. 

26. An optical disk reproduction apparatus according to 
claim 24, wherein: 5 

the reproduction control information includes 
timing information representing the timing to 
start decoding the leading second GOP so that 
reproduction completion time of the first picture 10 
matches the reproduction start time of the sec- 
ond picture, and 

the decoding section starts decoding the sec- 
ond video stream based on the timing informa- 
tion. 15 

27. An optical disk reproduction apparatus according to 
claim 23, wherein the decoding section omits 
decoding of a picture which is not necessary for 
decoding pictures from the leading picture of the 20 
leading second GOP to the second picture. 

28. An optical disk reproduction apparatus according to 
claim 27, wherein the picture which is not neces- 
sary is a B picture. 25 

29. An optical disk reproduction apparatus according to 
claim 23, further comprising a buffer memory sec- 
tion for storing the first video stream and the second 
video stream, and the buffer memory section has a 30 
capacity which is equal to or greater than an 
amount of data of 1 GOP. 

30. An optical disk reproduction apparatus according to 
claim 23, wherein the reproduction section repro- 35 
duces the reproduction control information 
recorded on the optical disk. 

31. An optical disk reproduction apparatus according to 
claim 23, wherein: 40 

the optical disk further has, recorded thereon, 
an identifier representing whether or not the 
reproduction control information is recorded on 
the optical disk, and 45 
when the identifier represents that the repro- 
duction control information is recorded on the 
optical disk, the reproduction section repro- 
duces the reproduction control information 
recorded on the optical disk. 50 

32. An optical disk reproduction apparatus according to 
claim 23, wherein in a fast reproduction mode, 
when the second picture is not an I picture, the out- 
put section prohibits an I picture included in the 55 
leading second GOP from being output. 

33. An optical disk reproduction apparatus according to 



claim 32, wherein the output section prohibits a part 
of the I picture included in the leading second GOP 
from being output based on I picture reproduction 
prohibition information. 

34. A reproduction control information generation 
apparatus, comprising: 

an input section for receiving a first video 
stream including a plurality of first GOPs and a 
second video stream including a plurality of 
second GOPs; and 

a generation section for generating reproduc- 
tion control information which represents that 
after a first picture included in a final first GOP 
among the plurality of first GOPs included in 
the first video stream is reproduced, a second 
picture included in a leading second GOP 
among the plurality of second GOPs included 
in the second video stream is reproduced, the 
second picture being different from a leading 
picture of the leading second GOP. 

35. A reproduction control information generation 
apparatus according to claim 34, wherein the repro- 
duction control information includes information 
representing the number of pictures from the lead- 
ing picture of the leading second GOP to the sec- 
ond picture. 

36. A reproduction control information generation 
apparatus according to claim 34, wherein the repro- 
duction control information includes information 
representing the time to reproduce the leading pic- 
ture of the leading second GOP and the time to 
reproduce the second picture of the leading second 
GOP. 

37. A reproduction control information generation 
apparatus according to claim 34, wherein the repro- 
duction control information includes timing informa- 
tion representing the timing to start decoding the 
leading second GOP so that reproduction comple- 
tion time of the first picture matches the reproduc- 
tion start time of the second picture. 

38. A reproduction control information generation 
apparatus according to claim 37, wherein the timing 
information represents the timing to start decoding 
the leading second GOP when a picture which is 
not necessary for decoding pictures from the lead- 
ing picture of the leading second GOP to the sec- 
ond picture is not decoded. 

39. A reproduction control information generation 
apparatus according to claim 38, wherein the pic- 
ture which is not necessary is a B picture. 
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40. An optical disk recording apparatus, comprising: 

a generation section for generating reproduc- 
tion control information; and 

a recording section for recording the reproduc- 5 
tion control information on an optical disk hav- 
ing, recorded thereon, a first video stream 
including a plurality of first GOPs and a second 
video stream including a plurality of second 
GOPs, 10 
wherein the reproduction control information 
represents that after a first picture included in a 
final first GOP among the plurality of first GOPs 
included in the first video stream is reproduced, 
a second picture included in a leading second 15 
GOP among the plurality of second GOPs 
included in the second video stream is repro- 
duced, the second picture being different from 
a leading picture of the leading second GOP. 

20 

41. An optical disk recording apparatus, comprising: 



45. An optical recording apparatus according to claim 
41, wherein the recording section records the 
edited first video stream and the edited second 
video stream in continuous regions of the optical 
disk. 

46. An optical recording apparatus according to claim 
41, wherein the recording section records the 
reproduction control information on the optical disk. 

47. An optical recording apparatus according to claim 
41, wherein the recording section records the 
reproduction control information on a medium other 
than the optical disk. 



an editing section for editing a first video 
stream including a plurality of first GOPs and a 
second video stream including a plurality of 25 
second GOPs so that at least one picture 
unnecessary for reproduction is deleted; and 
a recording section for recording the edited first 
video stream and the edited second video 
stream on an optical disk, 30 
wherein the reproduction control information 
represents that after a first picture included in a 
final first GOP among the plurality of first GOPs 
included in the first video stream is reproduced, 
a second picture included in a leading second 35 
GOP among the plurality of second GOPs 
included in the second video stream is repro- 
duced, the second picture being different from 
a leading picture of the leading second GOP. 

40 

42. An optical recording apparatus according to claim 
41 , wherein the at least one picture unnecessary for 
reproduction includes a picture, of the first video 
stream, after the first picture, and a picture, of the 
second video stream, before the second picture. 45 



43. An optical recording apparatus according to claim 
42, wherein the at least one picture unnecessary for 
reproduction further includes a picture which is not 
necessary for decoding pictures from the leading so 
picture of the leading second GOP in the second 
video stream until the second picture. 



44. A reproduction control information generation 

apparatus according to claim 43, wherein the at 55 
least one picture unnecessary for reproduction is a 
B picture. 
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FIG. 26 



(AV synchronization method of 1st and 2nd decoders) 
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Synchro nization of 1s t and 2nd decod ers is canceled 
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1st and 2nd decoders are synchronized vertically and horizontally. 
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APTS of audio signal is set as initial value of STC 
of 1st and 2nd decoders. 
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Processing' of 1st decoder 
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some as APTS is output from 
decoder in synchronization with 
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so that picture of 2nd VPTS 
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FIG. 27 



Control of 1st buffer 



Maximum capacity value (=1ILB) of one 
of main interleave blocks is set 
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FIG. 28 

(1) Reproduction data of 
stream A 



(2) Reproduction data of 
stream B 



(3) Stream clock (SCR) 

Time of input from 
buffer to decoder 



(4) Audio APTS (sub picture) 

Reproduction time of 
audio decoder 



(5) System clock 
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(6) Video VPTS2 

Time of reproduction of 
2nd video decoder 



(7) Video VPTS1 

Time of reproduction of 
1st video decoder 
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FIG. 29 

[ Program chain group is reproduced. 
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FIG. 33 



Time T1 



Time T2 



(Seamless connectioTu) 



STC offset is calculated and set 
in STC offset synthesis section. 



STC switch 1 is switched to 
STC offset synthesis section. 



STC switch 2 is switched to 
STC offset synthesis section. 



Time T3 STC switches 3, 3' are switched 
Time T3' to STC offset synthesis section. 



STC switch 4 is switched to 
STC offset synthesis section. 



Time T4 



Value in STC offset synthesis 
section is set in STC. 



STC switches 1 to 4 
are switched to STC. 
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FIG. 38 Reference time signal 
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FIG. 45 vpts|a 
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